Continuum robots are increasingly used in medical applications and the master–slave-based architectures are still the most important mode of operation in human–machine interaction. However, the existing master control devices are not fully suitable for either the mechanical mechanism or the control method. This study proposes a brand-new, four-degree-of-freedom haptic joystick whose main control stick could rotate around a fixed point. The rotational inertia is reduced by mounting all powertrain components on the base plane. Based on the design, kinematic and static models are proposed for position perception and force output analysis, while at the same time gravity compensation is also performed to calibrate the system. Using a continuum-mechanism-based trans-esophageal ultrasound robot as the test platform, a master–slave teleoperation scheme with position–velocity mapping and variable impedance control is proposed to integrate the speed regulation on the master side and the force perception on the slave side. The experimental results show that the main accuracy of the design is within 1.6°. The workspace of the control sticks is −60° to 110° in pitch angle, −40° to 40° in yaw angle, −180° to 180° in roll angle, and −90° to 90° in translation angle. The standard deviation of force output is within 8% of the full range, and the mean absolute error is 1.36°/s for speed control and 0.055 N for force feedback. Based on this evidence, it is believed that the proposed haptic joystick is a good addition to the existing work in the field with well-developed and effective features to enable the teleoperation of continuum robots for medical applications.