A passively mode-locked fiber laser with a 1.03 GHz fundamental repetition rate and a low noise performance is demonstrated. The compact and robust laser operates at a central wavelength of 1553.9nm with a 3 dB spectral bandwidth of 7.9 nm and a temporal width of 550 fs. All the fibers and components used in the laser are polarization-maintaining (PM), so the output pulses arc linearly polarized. The degree of polarization of the optical pulses is measured to be 0.9994. The phase noise of the seventh harmonic (7.21 GHz) is measured and a low timing jitter of 11.7 fs is obtained by integrating the phase noise from 30 MHz down to 100 Hz. The noise performance is analyzed in detail, and the methods for further optimizing the timing jitter are also proposed. So far, we know this to be the first-reported low-noise all-PM erbium-doped fiber laser with a gigahertz level pulse repetition rate.