In the microanalysis of laser-induced breakdown spectroscopy, the influence of surface roughness on spectral stability and quantitative analysis capability was studied for the first time when the laser ablation crater diameter was approximately 10 μm. By using a microchip laser, the spectra were acquired on a set of carbon steel samples, whose roughness changed from 0.0392 μm to 0.5248 μm. Besides, the study found that the surface roughness of the sample neglected the effect on spectral stability via screening and analyzing the spectral data when the sample stage moved at a constant speed. Moreover, when the spectral collection was performed with the ablation crater overlap rate of approximately 75% and Fe line was used as the internal standard, the effect of sample surface roughness on the quantitative analysis of the spectrum was negligible, and the resulting calculated determination coefficients of the Mn, Si, Ni, V, Cr, and Cu elements all exceeded 0.993.