Background: Tauopathy is a primary neuropathological hallmark of Alzheimer's disease with a strong relationship to cognitive impairment. In the brain, tau aggregation is associated with the regulation of tau kinases and the binding ability of tau to microtubules. Objective: To explore the potential for using specific polygenic risk scores (PRSs), combining the genetic influences involved in tau-protein kinases and the tau-protein binding pathway, as predictors of tau pathology and cognitive decline in non-demented individuals. Methods: We computed a pathway-specific PRS using summary statistics from previous large-scale genome-wide association studies of dementia. We examined whether PRS is related to tau uptake in positron emission tomography (PET), tau levels, and the rate of tau level changes in cerebrospinal fluid (CSF). We further assessed whether PRS is associated with memory impairment mediated by CSF tau levels. Results: A higher PRS was related to elevated CSF tau levels and tau-PET uptake at baseline, as well as greater rates of change in CSF tau levels. Moreover, PRS was associated with memory impairment, mediated by increased CSF tau levels. The association between PRS and tau pathology was significant when APOE was excluded, even among females. However, the effect of PRS on cognitive decline appeared to be driven by the inclusion of APOE. Conclusion: The influence of genetic risk in a specific tau-related biological pathway may make an individual more susceptible to tau pathology, resulting in cognitive dysfunction in an early preclinical phase of the disease.