Spatio-Temporal Dual Affine Differential Invariant for Skeleton-based Action Recognition

TL;DR

This paper introduces a novel spatio-temporal dual affine differential invariant feature for skeleton-based action recognition, enhancing neural network generalization and achieving state-of-the-art results on large-scale datasets.

Contribution

The paper proposes the STDADI feature and a channel augmentation method to improve action recognition accuracy and generalization in skeleton-based models.

Findings

Achieves significant improvements on NTU-RGB+D and NTU-RGB+D 120 datasets.

Introduces a novel spatio-temporal invariant feature for skeleton trajectories.

Enhances neural network robustness through channel augmentation.

Abstract

The dynamics of human skeletons have significant information for the task of action recognition. The similarity between trajectories of corresponding joints is an indicating feature of the same action, while this similarity may subject to some distortions that can be modeled as the combination of spatial and temporal affine transformations. In this work, we propose a novel feature called spatio-temporal dual affine differential invariant (STDADI). Furthermore, in order to improve the generalization ability of neural networks, a channel augmentation method is proposed. On the large scale action recognition dataset NTU-RGB+D, and its extended version NTU-RGB+D 120, it achieves remarkable improvements over previous state-of-the-art methods.