Remote sensing images often suffer low contrast. Although many contrast enhancement methods have been proposed in recent literature, the efficiency and robustness of remote sensing image contrast enhancement is still a challenge. In this letter, a novel self-adaptive histogram compacting transform-based contrast enhancement method for remote sensing images is presented to meet with the requirements of automation, robustness, and efficiency in applications. First, the histogram of an input image is optimized into compact and continuous status with the constraints of the merging cost, the moderate global brightness, and the entropy contribution of gray levels. Then, a local remapping algorithm is proposed to catch more details during the course of gray extending with the linear stretch. Finally, a dual-gamma transform is proposed to enhance the contrast in both bright and black areas. Experimental and comparison results demonstrate that the proposed method yields better results than the state-of-the-art methods and maintains robustness in different cases. It provides an effective approach for remote sensing image automatic contrast enhancement.