Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformation

TL;DR

This paper introduces Expressive Keypoints with detailed hand and foot information for more accurate skeleton-based action recognition, utilizing a Skeleton Transformation strategy and Instance Pooling for multi-person scenarios, achieving superior results.

Contribution

It presents a novel fine-grained skeletal representation and a Skeleton Transformation method, enhancing recognition accuracy and efficiency in multi-person contexts.

Findings

Outperforms state-of-the-art on seven datasets

Improves recognition of subtle human actions

Efficiently models multi-person scenarios

Abstract

In the realm of skeleton-based action recognition, the traditional methods which rely on coarse body keypoints fall short of capturing subtle human actions. In this work, we propose Expressive Keypoints that incorporates hand and foot details to form a fine-grained skeletal representation, improving the discriminative ability for existing models in discerning intricate actions. To efficiently model Expressive Keypoints, the Skeleton Transformation strategy is presented to gradually downsample the keypoints and prioritize prominent joints by allocating the importance weights. Additionally, a plug-and-play Instance Pooling module is exploited to extend our approach to multi-person scenarios without surging computation costs. Extensive experimental results over seven datasets present the superiority of our method compared to the state-of-the-art for skeleton-based human action recognition. Code is available at https://github.com/YijieYang23/SkeleT-GCN.