Hierarchical Compositional Representations for Few-shot Action Recognition

TL;DR

This paper introduces a hierarchical compositional approach for few-shot action recognition, decomposing complex actions into sub-actions and SAS-actions, and using Earth Mover's Distance for similarity measurement, achieving state-of-the-art results.

Contribution

The paper proposes a novel hierarchical compositional representation method that effectively captures sub-action patterns for few-shot action recognition, addressing data scarcity issues.

Findings

Achieves state-of-the-art results on HMDB51, UCF101, and Kinetics datasets.

Effectively models fine-grained sub-actions for improved recognition.

Utilizes Earth Mover's Distance for accurate similarity measurement.

Abstract

Recently action recognition has received more and more attention for its comprehensive and practical applications in intelligent surveillance and human-computer interaction. However, few-shot action recognition has not been well explored and remains challenging because of data scarcity. In this paper, we propose a novel hierarchical compositional representations (HCR) learning approach for few-shot action recognition. Specifically, we divide a complicated action into several sub-actions by carefully designed hierarchical clustering and further decompose the sub-actions into more fine-grained spatially attentional sub-actions (SAS-actions). Although there exist large differences between base classes and novel classes, they can share similar patterns in sub-actions or SAS-actions. Furthermore, we adopt the Earth Mover's Distance in the transportation problem to measure the similarity between video samples in terms of sub-action representations. It computes the optimal matching flows between sub-actions as distance metric, which is favorable for comparing fine-grained patterns. Extensive experiments show our method achieves the state-of-the-art results on HMDB51, UCF101 and Kinetics datasets.