Synchronized and Fine-Grained Head for Skeleton-Based Ambiguous Action Recognition

TL;DR

This paper introduces SF-Head, a lightweight module that improves skeleton-based ambiguous action recognition by balancing spatial-temporal features and maintaining global context, leading to better differentiation of similar actions.

Contribution

The paper proposes SF-Head, a novel plug-and-play module that enhances GCN-based action recognition by balancing features and preserving global context, specifically targeting ambiguous actions.

Findings

Significant accuracy improvements on multiple datasets.

Effective differentiation of ambiguous actions like waving and saluting.

Enhanced spatial-temporal feature interaction.

Abstract

Skeleton-based action recognition using GCNs has achieved remarkable performance, but recognizing ambiguous actions, such as "waving" and "saluting", remains a significant challenge. Existing methods typically rely on a serial combination of GCNs and TCNs, where spatial and temporal features are extracted independently, leading to an unbalanced spatial-temporal information, which hinders accurate action recognition. Moreover, existing methods for ambiguous actions often overemphasize local details, resulting in the loss of crucial global context, which further complicates the task of differentiating ambiguous actions. To address these challenges, we propose a lightweight plug-and-play module called SF-Head, inserted between GCN and TCN layers. SF-Head first conducts SSTE with a Feature Redundancy Loss (F-RL), ensuring a balanced interaction. It then performs AC-FA, with a Feature Consistency Loss (F-CL), which aligns the aggregated feature with their original spatial-temporal feature. Experimental results on NTU RGB+D 60, NTU RGB+D 120, NW-UCLA and PKU-MMD I datasets demonstrate significant improvements in distinguishing ambiguous actions.