It is essential to build the effective appearance model for object tracking in computer vision. Most object trackers can be roughly divided into two categories according to the appearance model: the bounding box model and the patch model. The bounding box model cannot handle shape deformation and occlusion of the non-rigid moving object effectively. The patch model is prone to be disturbed by complex backgrounds. In this paper, we propose a robust multi-structured-patch appearance model to represent the target for object tracking. The proposed appearance model is aimed to exploit and identify reliable patches that can be tracked effectively through the whole tracking process. According to attention mechanism in biological vision system, a coarse-to-fine strategy is usually used to search the target. Therefore, the proposed appearance model is represented by robust patches in different sizes, in which the bigger patches search the rough region of the target and the smaller patches estimate the accurate location. Experimental results on OTB100 dataset show that the proposed method outperforms state-of-the-art trackers.