Along with rapid global population aging, the age-related cognitive disorders such as mild cognitive impairment (MCI) and dementia have posed a serious threat to public health, health care system, and sustainable economic and societal development of all countries. In this narrative review, we seek to summarize the major epidemiological studies from the life-course perspective that investigate the influence of genetic susceptibility [e.g., apolipoprotein (APOE) epsilon 4 allele] and intellectual or psychosocial factors (e.g., educational attainments and leisure activities) as well as their interactions on cognitive phenotypes in aging. Numerous population-based studies have suggested that early-life educational attainments and socioeconomic status, midlife work complexity and social engagements, late-life leisure activities (social, physical, and mentally-stimulating activities), certain personality traits (e.g., high neuroticism and low conscientiousness), and depression significantly affect late-life cognitive phenotypes. Furthermore, certain intellectual or psychosocial factors (e. g., leisure activities and depression) may interact with genetic susceptibility (e.g., APOE epsilon 4 allele) to affect the phenotypes of cognitive aging such that risk or beneficial effects of these factors on cognitive function may vary by carrying the susceptibility genes. Current evidence from the randomized controlled trials that support the cognitive benefits of cognitive training among cognitive healthy older adults remains limited. The cognitive reserve hypothesis has been proposed to partly explain the beneficial effects of lifetime intellectual and psychosocial factors on late-life cognitive function. This implies that, from a life-course perspective, preventive intervention strategies targeting multiple modifiable intellectual and psychosocial factors could interfere with clinical expression of cognitive disorders in old age and delay the onset of dementia syndrome, and thus, may help achieve healthy brain aging.