The conventional numerical solvers for partial differential equations encounter a formidable challenge, as their computational efficiency and accuracy are heavily contingent on grid size. Recently, machine learning (ML) has exhibited substantial promise in addressing partial differential equations. Nevertheless, substantial hurdles persist in practical applications. In this work, we endeavor to establish a deep learning framework founded on the Fourier neural operator (FNO) for resolving the intricacies of simulating real landslide dynamic processes. Our findings demonstrate that the current FNO approach adeptly replicates landslide dynamic processes and boasts exceptional computational efficiency. Additionally, it is noteworthy that this data-driven ML methodology can seamlessly incorporate data from other experimental sources or numerical simulation techniques. Consequently, this work underscores the significant potential of utilizing ML methodologies to supplant conventional numerical simulation methods. There are great challenges involved in leveraging machine learning methods to learn realistic physical dynamic processes. When it comes to the real landslide movement across intricate terrains, it is meaningful to validate the capacities of machine learning in tackling the complicated problem. This study aims to propose an innovative solution of modeling of landslide dynamic processes from a machine learning perspective. Here, we introduce a data-driven framework based on Fourier neural operator to predict the dynamic behavior of actual landslides. Following an exhaustive assessment, the superior performance of our suggested model in real landslide situations and its versatility in adapting to landslides across various geographical regions have been confirmed. This study explores a new approach to modeling landslide dynamic processes and highlights the great potential of data-driven approaches to address dynamic process challenges present in real physical world. The data-driven deep learning method based on FNO can achieve fast prediction of real landslide accumulation process The proposed data-driven method for predicting landslide dynamic processes can be extended to new areas after transfer learning We provide numerical datasets of landslide dynamics, which can serve as the foundational resources for ML-based landslide forecasting tasks