Shadows cast by the rotating black hole in perfect fluid matter are studied using analytical method. We consider three kinds of matter with standard equation of state in Cosmology: -1/3 (dark matter), 0 (dust) and 1/3 (radiation). The apparent shape of the shadow depends on the black hole spin a and the perfect fluid matter intensity k. We find that the shadow is a perfect circle in the non-rotating case (a = 0) and a distorted one in the rotating case (a not equal 0), similar to what is found for other black holes. Interestingly, different kinds of matter have different influences on the black hole shadow. Dark matter has the strongest effect, dust is the next and radiation is the weakest. By applying our result to the black hole Sgr A* at the center of the Milky Way, we find that the angular resolution required to distinguish among different kinds of matter around the black hole is not much higher than that of current astronomical instruments would be able to achieve in the near future. We propose that observing the black hole shadow provides a possibility of identifying the dominant matter near the black hole.