Dynamic numerical simulation of large-scale landslides with material point method (MPM) is becoming increasingly popular in the geotechnical engineering and natural hazard community. However, efficient and accurate simulation of the interaction between landslides and complex terrain which the sliding mass moving remains a significant challenge. This study uses a fixed Triangulated Irregular Network (TIN) model from a digital elevation model to represent complex terrain, and models the interaction between sliding mass and terrain with a contact algorithm between TIN meshes and material points. A novel local neighborhood search (LNS) scheme with bucket-sorting algorithm is proposed to improve the particle-to-face contact detection efficiency. The new search algorithm significantly enhances both the efficiency and accuracy of matching detection between the triangular mesh and material points. The correctness and capability in handling complex dynamic problems are demonstrated through two numerical cases. The catastrophic Xinmo landslide occurred in China in 2017 is also simulated to validate the method's capability in large-scale landslide simulation. The results demonstrate that the proposed method is both accurate and computationally efficient. Furthermore, its adaptability and simplicity allow for seamless integration into other high-performance MPM frameworks, facilitating the simulation of large-scale gravity-driven flows on complex terrain.