As one of the key steps of machine visual investigation, camera calibration is indispensable for obtaining an accurate 3D profile of measured objects and positioning the target. However, it is almost impossible to calibrate a camera on-orbit in real time, for the feature points in space are irregular. To self-calibrate camera parameters in orbit, we propose a method to calibrate the camera using the orthogonal vanishing points obtained from a solar panel, which is a common component of most man-made space satellites. Using two sets of images of orthogonal parallel ribs of a solar panel under any two positions, four vanishing points are achieved. Based on the geometrical property of the orthogonal vanishing point and the infinite homography relationship between the corresponding vanishing points under different camera positions, a camera self-calibration method is proposed, and constraint equations are established to calibrate the intrinsic parameters of the camera linearly. Aiming at restraining the influence of the noise on the calibration result, an objective function based on the inverse point characteristics of rectangular imaging is proposed, and the Levenberg-Marquardt optimization algorithm is used to nonlinearly optimize the intrinsic parameters. Simulated and experimental results show that the 2D reprojection error is 0.86 pixels. This method, therefore, has better applicability in some significant space and national defense missions, such as space rendezvous and docking, attack-defense, and on-orbit servicing. The optimization algorithm shows good anti-noise interference performance, therefore the robustness of the calibration algorithm is improved.