Image matching is an important topic in the field of computer vision, in view of high robustness and accuracy, SIFT or the improved methods based on SIFT is generally used for image matching algorithms. The traditional SIFT method is implemented on grayscale images without regard to the color information of images, which may cause decreasing of the matching points and reduction of the matching accuracy. Prevailing color descriptors can effectively add color information into SIFT, however dramatically increase the complexity of algorithm. In this paper, a novel approach is proposed to take advantage of the color information for image matching based on SIFT. The proposed algorithm uses the gradient information of color channel as the compensation of luminance channel, which can effectively enhance the color information with SIFT. Experimental results show that the number of feature points and matching accuracy can be significantly promoted, while the complexity and performance of image matching algorithm are well trade-off. © 2018 SPIE.