As a large weathered granite soil region in the world, southern China is experiencing severe sheet and rill erosion and gully erosion due to its special geological conditions, inappropriate land use, and high intensity precipitation. However, few studies have been conducted on the long-term dynamics of these two types of soil erosion and the sediment contribution of gully erosion in southern China. To assess this, estimates of soil erosion changes and the contribution ratio of gully erosion to sediment yield in a small catchment in southern China from 1989 to 2015 are quantified. A topographic map and three high-resolution satellite images were interpreted for land use changes to calculate soil erosion modulus of sheet and rill erosion by the Chinese soil loss equation (CSLE) and to estimate the amount of gully erosion by gully volume based on a generalized inverted triangular pyramid model multiplying bulk density. Results indicated that soil erosion modulus started as high as 10,442 t km−2 in 1989, followed by an initial decrease of 54% before increasing slightly. The total gully volume decreased by 37% overall, as did the cumulative amount of gully erosion. Between 2005 and 2010 positive total soil loss was calculated and the sediment contribution ratio of gully erosion was estimated to be 52.27%. It can be concluded that the combination of remote sensing images, the CSLE, and a new generalized gully model can effectively be used to quantify soil loss from sheet and rill erosion as well as gully erosion. This study provides a new insight to estimate gully erosion and the dynamic evolution of soil loss. Further studies should be performed in combination with field methods to validate this approach.