One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching

TL;DR

This paper introduces a multi-scale spatial-temporal feature matching approach for one-shot skeleton action recognition, effectively capturing spatial and temporal structures to improve recognition accuracy.

Contribution

It proposes a novel multi-scale and cross-scale feature matching method that considers spatial and temporal hierarchies, advancing one-shot skeleton action recognition.

Findings

Outperforms state-of-the-art on NTU RGB+D datasets

Achieves superior accuracy in one-shot recognition tasks

Demonstrates robustness across different motion magnitudes and speeds

Abstract

One-shot skeleton action recognition, which aims to learn a skeleton action recognition model with a single training sample, has attracted increasing interest due to the challenge of collecting and annotating large-scale skeleton action data. However, most existing studies match skeleton sequences by comparing their feature vectors directly which neglects spatial structures and temporal orders of skeleton data. This paper presents a novel one-shot skeleton action recognition technique that handles skeleton action recognition via multi-scale spatial-temporal feature matching. We represent skeleton data at multiple spatial and temporal scales and achieve optimal feature matching from two perspectives. The first is multi-scale matching which captures the scale-wise semantic relevance of skeleton data at multiple spatial and temporal scales simultaneously. The second is cross-scale matching which handles different motion magnitudes and speeds by capturing sample-wise relevance across multiple scales. Extensive experiments over three large-scale datasets (NTU RGB+D, NTU RGB+D 120, and PKU-MMD) show that our method achieves superior one-shot skeleton action recognition, and it outperforms the state-of-the-art consistently by large margins.