To ensure safe and efficient autonomous driving, trajectory prediction system must account for social interactions among road participants. Graph-based models are leading approaches in modeling social interactions for trajectory prediction, but they face the challenges of designing an appropriate graph structure and processing complex interactions. We consider that the participants in a scene are a set of unstructured points, which are similar to point cloud data. Inspired by
point cloud learning networks, we view the road participants in a scene as point cloud in a two-dimensional coordinate system, and utilize Point Transformer aggregator to process the interactions on both local and global level. Besides, we present a multiplex fusion of social and temporal information for trajectory prediction. We perform extensive experiments
on the Argoverse motion forecasting dataset, and the results demonstrate the superior performance of our model for multi-agent trajectory prediction.