Developing expertise in any field usually requires acquisition of a wide range of skills. Most current studies on perceptual learning have focused on a single task and concluded that learning is quite specific to the trained task, and the ubiquitous individual differences reflect random fluctuations across subjects. Whether there exists a general learning ability that determines individual learning performance across multiple tasks remains largely unknown. In a large-scale perceptual learning study with a wide range of training tasks, we found that initial performance, task, and individual differences all contributed significantly to the learning rates across the tasks. Most importantly, we were able to extract both a task-specific but subject-invariant component of learning, that accounted for 38.6% of the variance, and a subject-specific but task-invariant perceptual learning ability, that accounted for 36.8% of the variance. The existence of a general perceptual learning ability across multiple tasks suggests that individual differences in perceptual learning are not "noise"; rather, they reflect the variability of learning ability across individuals. These results could have important implications for selecting potential trainees in occupations that require perceptual expertise and designing better training protocols to improve the efficiency of clinical rehabilitation.