Social attention is crucial for efficient social interactions and adaptive functioning in humans. However, whether this indispensable ability is unique and qualitatively distinct from nonsocial attention remains equivocal. Using the visual adaptation technique in conjunction with a modified central cueing paradigm, the current study investigated the specificity of social attention. Results revealed that adaptation to the walking direction of biological motion (BM) affected the reflexive attentional effect triggered by subsequent BM cues. Critically, preexposure to another type of social cues (i.e., eye gaze) could produce a similar aftereffect on attentional orienting elicited by BM, reflecting that social attention induced by different types of cues might share common neural substrates. By contrast, such cross-category adaptation aftereffect disappeared when adaptors changed to nonsocial cues (i.e., arrows). In the same vein, adaptation to BM cues could also exert an aftereffect on gaze cueing but not arrow cueing effect. Taken together, these findings provide evidence for the view that "social attention is special" and support the existence of "social attention detector" in the human brain.