Understanding the effect of sunspot activities on the variations in the total solar irradiance (TSI) is essential for the interpretation of the variability of TSI as well as its reconstruction. Phase relations between the sunspot numbers (SN) and two TSI composite data are investigated. It is found that TSI and SN are positively correlated, and the former lags the latter by about 29 days, which is approximately a solar rotation period; analyses of the data sets in the four individual cycles show that in cycles 21, 23, and 24, TSI lags SN by 28.9-30.3 days, while in cycle 22, the lag is only 21.8-22.3 days. The abnormality in cycle 22 is probably caused by its stronger magnetic field in sunspots compared with its adjacent cycles. The nonlinearity between TSI and SN is confirmed and explained with the different behavior and effect of spots, faculae, and magnetic network. Based on the cross-wavelet transform and wavelet coherence analysis, a common periodicity between TSI and SN at the timescale of the solar cycle is clearly revealed; at timescales longer than about four years, high values of coherence above the 95% confidence level together with a strong phase synchronization feature are exhibited. At timescales shorter than three rotational periods, the relation between TSI and SN indicates low correlations and a noisy behavior with strong phase mixing due to the lifetime of spots and faculae; moreover, if the short-term effect of spots and faculae is smoothed out, then their coherence reaches high values in partial areas at periods from three rotations to about four years.