Magnetic particle imaging (MPI) is a tomographic imaging method that quantitatively determines the distribution of magnetic nanoparticles (MNPs). However, the performance of MPI is primarily limited by the noise in the receive coil and electronic devices, which causes quantification errors for MPI images. Existing methods cannot efficiently eliminate noise while preserve structural details in MPI images. To address this problem, we propose a Content-Noise Feature Fusion Neural Network equipped with tailored modules of noise learning and content learning. It can simultaneously learn content and noise features of raw MPI images. Experimental results show that the proposed method outperforms the state-of-the-art methods on structural details preservation and image noise reduction of different levels.