The successful resolution of ever-changing conflicting contexts requires efficient cognitive control. Previous studies have found similar neural patterns in conflict processing for different modalities using an event-related potential (ERP) approach and have concluded that cognitive control is supramodal. However, recent behavioral studies have found that conflict adaptation (a phenomenon with the reduction of congruency effect in the current trial after an incongruent trial as compared with a congruent trial) could not transfer across visual and auditory modalities and suggested that cognitive control is modality-specific, challenging the supramodal view. These discrepancies may have also arisen from methodological differences across studies. The current study examined the electroencephalographic profiles of a Stroop-like task to elucidate the modality-specific neural mechanisms of cognitive control. Participants were instructed to respond to a target always coming from the visual modality while disregarding the distractor coming from either the auditory or the visual modality. The results revealed significant congruency effects on both behavioral indices, i.e., reaction time and error rate, and ERP components, including the P3 and the conflict slow potential. Besides, the congruency effects on the amplitude of the P3 showed a negative correlation with reaction time, indicating an intrinsic link between these neural and behavioral indices. Furthermore, in the modality-repetition condition, conflict adaptation effects were significant on both reaction time and P3 amplitude, and the reaction time could be predicted by the P3 amplitude, while such effects were not observed in the modality-alternation condition. The time-frequency analysis also showed that conflict adaptation occurred in the modality-repetition condition, but not in the modality-alternation condition in low frequency bands, including the theta (4-8 Hz), alpha (8-12 Hz), and beta1 (12-20 Hz) bands. Taken together, our results revealed modality-specific patterns of the conflict adaptation effects on the P3 amplitude and oscillatory power (in theta, alpha, and beta1 bands), providing neural evidence for the modality specificity of cognitive control and expanding the boundaries of cognitive control.