Shape memory alloy (SMA) has been utilized as the material of smart actuators due to the miniaturization and lightweight. However, the nonlinearity and hysteresis of SMA material seriously affect the precise control. In this article, a novel disturbance compensation-based adaptive control scheme is developed to improve the control performance of SMA actuator system. Firstly, the nominal model is constructed based on the physical process. Next, an estimator is developed to online update not only the unmeasured system states but also the total disturbance. Then, the novel adaptive controller, which is composed of the nominal control law and the compensation control law, is designed. Finally, the proposed scheme is evaluated in the SMA experimental setup. The comparison results have demonstrated that the proposed control method can track reference trajectory accurately, reject load variations and stochastic disturbances timely, and exhibit satisfactory robust stability. The proposed control scheme is system independent and has some potential in other types of SMA-actuated systems.