Ship detection in high-resolution optical remote sensing images (ORSI) is an important yet challenging task with extensive applications, such as maritime security and resource conservation. In recent years, bolstered by deep learning, ship detection has also grown by leaps and bounds. Nevertheless, existing methods still suffer from two challenging issues: 1) imprecise localization for low discriminative ships under the complicated background and 2) missed detections for small ships. To solve the above issues, we propose a novel ship detection method equipped with a saliency-guided feature fusion network (SGFFN) and a dynamic IoU-adaptive strategy (DIAS). SGFFN is designed based on a top-down feature pyramid network to introduce saliency information into the ship detection network and optimize the saliency-aware features. It comprises two components: the resolution-matching saliency supervision (RMS) network and the cross-stage saliency integration network (CSIN). RMS is a bimatching mechanism that adopts diverse prediction structures for the saliency maps with different scales, such that the finer saliency-aware features could be obtained. CSIN is a cross-stage cross-channel integration module that is designed to fuse saliency-aware features with low-level features. Furthermore, a customized training strategy for small ships, i.e., DIAS, is devised to assign appropriate intersection over union (IoU) thresholds for anchors around the small ships during the training phase. Experimental results on two datasets demonstrate that our proposed method achieves state-of-the-art performance.