Due to advantages such as the high efficiency of light utilization, small volume, and vibration resistance, digital micro-mirror device (DMD)-based spectrometers are widely used in ocean investigations, mountain surveys, and other field science research. In order to eliminate the stray light caused by DMDs, the stray light in DMD-based spectrometers was first measured and analyzed. Then, the stray light was classified into wavelength-related components and wavelength-unrelated components. Moreover, the noise caused by the stray light was analyzed from the perspective of encoding equation, and the de-noising decoding equation was deduced. The results showed that the accuracy range of absorbance was enhanced from [0, 1.9] to [0, 3.1] in single-stripe mode and the accuracy range of absorbance was enhanced from [0, 3.8] to [0, 6.3] in Hadamard transform (HT) multiple-stripe mode. A conclusion can be drawn that the de-noising strategy is feasible and effective for enhancing the SNR in DMD-based spectrometers.