Infrared small-dim target detection is an important technology in the fields of infrared guidance, anti-missile, and tracking system. Due to the small size of targets, no obvious structure information, and low image signal-to-noise ratio, infrared small-dim target detection is still a challenging task. In this letter, a cross-connected bidirectional pyramid network (CBP-Net) is proposed for infrared small-dim target detection. The main body of the CBP-Net is to embed a bottom-up pyramid in the Feature Pyramid Network (FPN), which is designed to provide more comprehensive target information by connecting with the original multi-scale features and the top-down pyramid. The bottom-up pyramid together with the top-down pyramid forms the proposed bidirectional pyramid structure. Then, an ROI feature augment module composed of deformable ROI pooling and position attention is designed to fuse multi-scale ROI features and enhance the spatial information of the small-dim target. Besides, a regular constraint loss is introduced to restrict multi-scale feature fusion to learn more precise target location information. Experimental results on two challenging datasets show that the performance of the proposed CBP-Net is superior to the state-of-the-art methods. IEEE