Strengthening Skeletal Action Recognizers via Leveraging Temporal Patterns

TL;DR

This paper introduces two temporal components, DCE and CRL, to enhance skeleton-based action recognizers by capturing frequency domain features and sequence order, leading to improved accuracy and state-of-the-art results.

Contribution

The work proposes two novel temporal accessories, DCE and CRL, that are compatible with existing models to boost their performance in skeleton-based action recognition.

Findings

Achieved new state-of-the-art accuracy on two large datasets.

DCE helps analyze motion in the frequency domain and reduces noise influence.

CRL explicitly captures the chronological order of sequences.

Abstract

Skeleton sequences are compact and lightweight. Numerous skeleton-based action recognizers have been proposed to classify human behaviors. In this work, we aim to incorporate components that are compatible with existing models and further improve their accuracy. To this end, we design two temporal accessories: discrete cosine encoding (DCE) and chronological loss (CRL). DCE facilitates models to analyze motion patterns from the frequency domain and meanwhile alleviates the influence of signal noise. CRL guides networks to explicitly capture the sequence's chronological order. These two components consistently endow many recently-proposed action recognizers with accuracy boosts, achieving new state-of-the-art (SOTA) accuracy on two large datasets.