Optical instrumentation with reliable performance is essential for the research of exoplanet atmosphere characteristics. However, due to long distances and weak signals, exoplanets are difficult to be imaged by traditional optical systems. To this end, a novel optical system based on transit spectroscopy is proposed in this paper. On the basis of the principle of the transit-spectroscopy method and the astronomical parameters of observed targets, the optional parameter ranges of a dedicated optical system are analyzed. The transit signal-to-noise ratio (SNR) is introduced for the determination of telescope aperture and throughput. Furthermore, an example of the optical system with a space telescope and spectrometer is proposed according to the above optical index, which is proven to meet the performance requirements. The optical system is required to cover the wavelength of 0.5-8 mu m and the field of view (FOV) of 27.9 '' within the diffraction limit. The collecting aperture should be greater than 2 m, and spectral resolutions of two spectrometer channels should approximately be 100 (2-4 mu m) and 30 (4-8 mu m). The point-spread function (PSF) of each channel at the minimal wavelength should cover 2 pixels. The telescope and dichroic system provide diffraction-limited input beams with the required aperture, FOV, and wavelength for the spectrometer slits. The simulation results of the optical system show that the spectral resolutions of the dual-channel spectrometer were 111-200 and 43-94. The image points of the spectrometer in each wavelength were smaller than the Airy spot within the slit FOV, and the full width at half-maximum (FWHM) of PSF at lambda(min) provided 2 pixels of 18 mu m sampling. The feasibility of the demonstrated optical parameters is proven by the design.