Aiming at the problem of tracking delay and large errors when conventional image-based visual servo is applied to tracking moving objects, a velocity compensation image-based visual servo controller is proposed for head-fixed oculomotor control in this paper, which covers saccade, smooth pursuit and vergence. The controller consists of a basic visual servo sub-controller and a velocity compensation sub-controller. The former is used to eliminate position error and the latter takes into account the target's velocity. Corresponding Jacobian matrixes are derived to implement the controller. At the same time, a novel adaptive gain is designed to boost the control law and continuous velocities are implemented to avoid abrupt changes. A simple but stable fixation point detection method is proposed to provide the input for the whole system. Extensive experiments are conducted and analysed in a real binocular platform implemented with off-the-shelf set-ups, which demonstrate the effectiveness of the proposed method.