Given the key role of pitch and time in the mental representation of music, the way the two dimensions combine is a crucial question in music cognition. In the present study, using electroencephalography (EEG), we manipulated both musical pitch and time structures and investigated how the two dimensions work together. Musicians were presented with eight-chord sequences, in which the last target chord was harmonically or temporally expected or unexpected based on the preceding contexts. ERP analysis showed that listeners track both dimensions as music unfolds in time. For the time dimension, irregular temporal events induced greater MMN than regular temporal events. For the pitch dimension, harmonically less-related chords revealed greater ERAN and N5 than harmonically related chords. Moreover, there was an interaction between musical pitch and time dimensions in the N5 effect. These results indicate that for music perception, pitch and time dimensions are processed independently at the early stage and interactively at the late stage.