MicroRNAs (miRNAs), critical biomarkers of acute and chronic diseases, play key regulatory roles in many biological processes. As a result, there is great demand for robust assay platforms to enable an accurate and efficient detection of low-level miRNAs in complex biological samples. In this work, a label-free and Au nanoparticles (NPs) quenching-based competition assay system was developed. In the designed system, Au NPs with diameter sizes of 10 and 20 nm displayed fluorescence quenching efficiencies of 84% and 82% for Cy3 dye on slide surface, whereas the quenching efficiency of commercial BHQ2 quencher was roughly 50%. Assay conditions were optimized for miRNA-205 detection. A limit of detection of 3.8 pM and a detection range covering from 3.8 pM to 10 nM were achieved. Furthermore, the proposed system was capable of specifically discriminating miRNAs with slight variations in their nucleotide sequence and was also qualified for assessing miRNA levels in human serum. Our strategy has the potential to provide new perspectives in profiling the pattern of miRNA expression and biomedical utilizations.