Understanding functional connectivity of the amygdala with other brain regions, especially task modulated connectivity, is a critical step toward understanding the role of the amygdala in emotional processes and the interactions between emotion and cognition. The present study performed coordinate-based meta-analysis on studies of task modulated connectivity of the amygdala which used psychophysiological interaction (PPI) analysis. We first analyzed 49 PPI studies on different types of tasks using activation likelihood estimation (ALE) meta-analysis. Widespread cortical and subcortical regions showed consistent task modulated connectivity with the amygdala, including the medial frontal cortex, bilateral insula, anterior cingulate, fusiform gyrus, parahippocampal gyrus, thalamus, and basal ganglia. These regions were in general overlapped with those showed coactivations with the amygdala, suggesting that these regions and amygdala are not only activated together, but also show different levels of interactions during tasks. Further analyses with subsets of PPI studies revealed task specific functional connectivities with the amygdala that were modulated by fear processing, face processing, and emotion regulation. These results suggest a dynamic modulation of connectivity upon task demands, and provide new insights on the functions of the amygdala in different affective and cognitive processes. The meta-analytic approach on PPI studies may offer a framework toward systematical examinations of task modulated connectivity.