Dynamic Spatial-temporal Hypergraph Convolutional Network for Skeleton-based Action Recognition

TL;DR

This paper introduces DST-HCN, a novel neural network that captures complex spatial-temporal dependencies in skeleton data for action recognition, achieving state-of-the-art results on multiple datasets.

Contribution

It proposes a dynamic spatial-temporal hypergraph convolutional network with a time-point hypergraph and high-order information fusion for improved skeleton-based action recognition.

Findings

Achieves state-of-the-art accuracy on NTU RGB+D datasets.

Effectively models spatial-temporal dependencies with hypergraphs.

Outperforms existing hypergraph-based methods.

Abstract

Skeleton-based action recognition relies on the extraction of spatial-temporal topological information. Hypergraphs can establish prior unnatural dependencies for the skeleton. However, the existing methods only focus on the construction of spatial topology and ignore the time-point dependence. This paper proposes a dynamic spatial-temporal hypergraph convolutional network (DST-HCN) to capture spatial-temporal information for skeleton-based action recognition. DST-HCN introduces a time-point hypergraph (TPH) to learn relationships at time points. With multiple spatial static hypergraphs and dynamic TPH, our network can learn more complete spatial-temporal features. In addition, we use the high-order information fusion module (HIF) to fuse spatial-temporal information synchronously. Extensive experiments on NTU RGB+D, NTU RGB+D 120, and NW-UCLA datasets show that our model achieves state-of-the-art, especially compared with hypergraph methods.