In skeleton-based action recognition, it has been a
dominant paradigm to extract motion features with temporal convolution
and model spatial correlations with graph convolution.
However, it’s difficult for temporal convolution to capture longrange
dependencies effectively. Meanwhile, commonly used multibranch
graph convolution leads to high complexity. In this paper,
we propose TranSkeleton, a powerful Transformer framework
which neatly unifies the spatial and temporal modeling of skeleton
sequences. For temporal modeling, we propose a novel partitionaggregation
temporal Transformer. It works with hierarchical
temporal partition and aggregation, and can capture both longrange
dependencies and subtle temporal structures effectively.
A difference-aware aggregation approach is designed to reduce
information loss during temporal aggregation. For spatial modeling,
we propose a topology-aware spatial Transformer which
utilizes the prior information of human body topology to facilitate
spatial correlation modeling. Extensive experiments on two
challenging benchmark datasets demonstrate that TranSkeleton
notably outperforms the state of the arts.