Both pictures and words are frequently employed as experimental stimuli to investigate the neurocognitive mechanisms of emotional processing. However, it remains unclear whether emotional picture processing and emotional word processing share neural underpinnings. To address this issue, we focus on neuroimaging studies examining the implicit processing of affective words and pictures, which require participants to meet cognitive task demands under the implicit influence of emotional pictorial or verbal stimuli. A coordinate-based activation likelihood estimation meta-analysis was conducted on these studies, which revealed no common activation maximum between the picture and word conditions. Specifically, implicit negative picture processing (35 experiments, 393 foci, and 932 subjects) engages the bilateral amygdala, left hippocampus, fusiform gyri, and right insula, which are mainly located in the subcortical network and visual network associated with bottom-up emotional responses. In contrast, implicit negative word processing (34 experiments, 316 foci, and 799 subjects) engages the default mode network and fronto-parietal network including the ventrolateral prefrontal cortex, dorsolateral prefrontal cortex, and dorsomedial prefrontal cortex, indicating the involvement of top-down semantic processing and emotion regulation. Our findings indicate that affective pictures (that intrinsically have an affective valence) and affective words (that inherit the affective valence from their object) modulate implicit emotional processing in different ways, and therefore recruit distinct brain systems.