Autonomous docking guidance is one of the key technologies to achieve the autonomous underwater vehicle (AUV) docking with the sub-sea docking station (DS) to realize long-term resident operation. In the process of AUV docking, the combination of long-distance acoustic guidance based on acoustic sensor and terminal visual guidance based on camera is often adopted. However, affected by the accuracy of the navigation sensor and acoustic positioning sensor carried by AUV, as well as the ocean current, AUV cannot accurately know its own position and the position of the DS, resulting in a large acoustic guidance error and the inability to enter the visual guidance stage with a reasonable deviation, thus leading to the docking failure. In this article, an improved FastSLAM algorithm is proposed to estimate the position of AUV and DS simultaneously. The positioning accuracy of traditional FastSLAM algorithm is affected by such factors as the estimation accuracy of the statistical characteristics of process noise. An improved algorithm for FastSLAM based on fuzzy Q-learning is proposed. The homing path is planned based on the Dubins theory. The path is tracked by line-of-sight guidance. The results of matlab simulation and experimental data analyzing of the portable AUV are applied to verify the effectiveness of the proposed algorithm.