Spatial-Temporal Alignment Network for Action Recognition

TL;DR

This paper introduces the Spatial-Temporal Alignment Network (STAN), a lightweight and generic module that enhances existing action recognition models by learning geometric invariant features, improving accuracy on standard datasets.

Contribution

The paper proposes a novel STAN module that can be integrated into existing models to handle geometric variations, with minimal additional computational cost.

Findings

STAN improves accuracy on UCF101 and HMDB51 datasets.

STAN is lightweight and easily plug-in to existing models.

Experimental results show consistent performance gains.

Abstract

This paper studies introducing viewpoint invariant feature representations in existing action recognition architecture. Despite significant progress in action recognition, efficiently handling geometric variations in large-scale datasets remains challenging. To tackle this problem, we propose a novel Spatial-Temporal Alignment Network (STAN), which explicitly learns geometric invariant representations for action recognition. Notably, the STAN model is light-weighted and generic, which could be plugged into existing action recognition models (e.g., MViTv2) with a low extra computational cost. We test our STAN model on widely-used datasets like UCF101 and HMDB51. The experimental results show that the STAN model can consistently improve the state-of-the-art models in action recognition tasks in trained-from-scratch settings.