Based on the saliency detection which effectively extracts the saliency information of the image and the adaptive double-channel spiking cortical model (ADSCM) which has the advantages of global coupling, pulse synchronization, less parameters and high computational efficiency, which can well process information in dark areas, we proposed a new infrared and visible image fusion algorithm. Firstly, the infrared and visible images are decomposed into the base layer and the detail layers through the rolling guidance filter (RGF). Secondly, the base layer is fused with an improved saliency detection method, and the detail layers are fused by a directional gradient sum and an improved ADSCM. The experimental results show that the proposed method can also fully extract the target information of the infrared image while preserving the visible background information, in which the fusion effect is better than the traditional fusion algorithms. © 2019 Elsevier B.V.