Bed friction characteristics are important factors controlling landslide movement processes. To address existing boundary condition deficiencies in describing the bed friction characteristics of landslide movements on complex terrain for the SPH method, a modified friction-viscous solid boundary condition is proposed. Within the solid boundary condition, the solid boundary is composed of line segments in 2D conditions and triangular segments in 3D conditions. The solid boundary is treated as a rigid material, and the SPH particle is treated as an elastic particle. The method used to calculate the values and directions of the contact, friction and viscous forces is presented. The proposed solid boundary condition is validated through water and granular material column collapse experiments. The dynamic process of a realistic landslide on complex terrain is simulated to demonstrate the applicability of the proposed solid boundary condition. Compared with the existing solid boundary condition, the proposed solid boundary condition has advantages in embedding friction model and wall function, exerting friction and viscosity force, keeping the stability of contact force, and establishing a complex terrain model.