TSGCNeXt: Dynamic-Static Multi-Graph Convolution for Efficient Skeleton-Based Action Recognition with Long-term Learning Potential

TL;DR

TSGCNeXt introduces a simplified, efficient multi-graph convolutional network for skeleton-based action recognition, significantly improving speed and accuracy on large-scale datasets by modeling long-term temporal features.

Contribution

It proposes a novel Dynamic-Static Separate Multi-graph Convolution mechanism and a training acceleration method, enhancing long-term skeleton sequence learning and computational efficiency.

Findings

Outperforms existing methods on NTU RGB+D 60 and 120 datasets.

Achieves state-of-the-art accuracy with multi-stream fusion.

Speeds up dynamic graph learning by 55.08%.

Abstract

Skeleton-based action recognition has achieved remarkable results in human action recognition with the development of graph convolutional networks (GCNs). However, the recent works tend to construct complex learning mechanisms with redundant training and exist a bottleneck for long time-series. To solve these problems, we propose the Temporal-Spatio Graph ConvNeXt (TSGCNeXt) to explore efficient learning mechanism of long temporal skeleton sequences. Firstly, a new graph learning mechanism with simple structure, Dynamic-Static Separate Multi-graph Convolution (DS-SMG) is proposed to aggregate features of multiple independent topological graphs and avoid the node information being ignored during dynamic convolution. Next, we construct a graph convolution training acceleration mechanism to optimize the back-propagation computing of dynamic graph learning with 55.08\% speed-up. Finally, the TSGCNeXt restructure the overall structure of GCN with three Spatio-temporal learning modules,efficiently modeling long temporal features. In comparison with existing previous methods on large-scale datasets NTU RGB+D 60 and 120, TSGCNeXt outperforms on single-stream networks. In addition, with the ema model introduced into the multi-stream fusion, TSGCNeXt achieves SOTA levels. On the cross-subject and cross-set of the NTU 120, accuracies reach 90.22% and 91.74%.