Neuroimaging studies have consistently indicated that semantic processing involves a brain network consisting of multimodal cortical regions distributed in the frontal, parietal, and temporal lobes. However, little is known about how semantic information is organized and processed within the network. Some recent studies have indicated that sensorymotor semantic information modulates the activation of this network. Other studies have indicated that this network responds more to social semantic information than to other information. Using fMRI, we collectively investigated the brain activations evoked by social and sensory-motor semantic information by manipulating the sociality and imageability of verbs in a word comprehension task. We detected 2 subgroups of brain regions within the network showing sociality and imageability effects, respectively. The 2 subgroups of regions are distinct but overlap in bilateral angular gyri and adjacent middle temporal gyri. A follow-up analysis of resting-state functional connectivity showed that dissociation of the 2 subgroups of regions is partially associated with their intrinsic functional connectivity differences. Additionally, an interaction effect of sociality and imageability was observed in the left anterior temporal lobe. Our findings indicate that the multimodal cortical semantic network has fine subdivisions that process and integrate social and sensory- motor semantic information.