Radio astronomy plays a very important role in promoting scientific progress and unraveling the mysteries of the universe. However, radio telescopes are inevitably affected by radio frequency interference (RFI) when receiving radio signals, which leads to a reduction in data quality and has a serious impact on the formation of correct scientific conclusions. Therefore, it is essential to identify the RFI present in the observational data. In order to effectively identify RFI, improve the existing RFI identification methods that suffer from missed detections, and enhance the performance of RFI identification, this paper proposes a novel method that combines a dual cross-attention mechanism with multi-scale feature fusion. Experimental studies were conducted using the observational data from the 40-meter radio telescope at the Yunnan Astronomical Observatory of the Chinese Academy of Sciences. The proposed method achieved scores of 92.49%, 83.90%, and 87.99% in terms of precision, recall and F1-score, respectively. It outperformed existing methods (U-Net, RFI-Net, R-Net6, RFI-GAN, EMSCA-UNet) in recall and F1-score, effectively reducing the occurrence of missed detections and improving the overall performance of radio frequency interference identification.