Understanding the contact properties of graphene on semiconductors is crucial to improving the performance of graphene optoelectronic devices. Here, we show that when graphene is in contact with a semiconductor, the charge carrier transport into graphene leads to a self-adaptive shift of the Fermi level, which tends to lower the barrier heights of the graphene contact to both n- and p-type semiconductors. A theoretical model is presented to describe the charge carrier transport mechanism and to quantitatively estimate the barrier heights. These results can benefit recent topical approaches for graphene integration in various semiconductor devices.