AFE-CNN: 3D Skeleton-based Action Recognition with Action Feature Enhancement

TL;DR

The paper introduces AFE-CNN, a lightweight model that enhances 3D skeleton action features to improve recognition accuracy, especially for challenging actions, while reducing computational costs.

Contribution

It proposes novel feature enhancement modules from multiple perspectives and employs a lightweight CNN, addressing limitations of handcrafted features and heavy models in existing methods.

Findings

Achieves superior accuracy on NTU RGB+D, NTU RGB+D 120, and UTKinect-Action3D datasets.

Demonstrates robustness to camera view and body size variations.

Ensures lower computational burden compared to state-of-the-art methods.

Abstract

Existing 3D skeleton-based action recognition approaches reach impressive performance by encoding handcrafted action features to image format and decoding by CNNs. However, such methods are limited in two ways: a) the handcrafted action features are difficult to handle challenging actions, and b) they generally require complex CNN models to improve action recognition accuracy, which usually occur heavy computational burden. To overcome these limitations, we introduce a novel AFE-CNN, which devotes to enhance the features of 3D skeleton-based actions to adapt to challenging actions. We propose feature enhance modules from key joint, bone vector, key frame and temporal perspectives, thus the AFE-CNN is more robust to camera views and body sizes variation, and significantly improve the recognition accuracy on challenging actions. Moreover, our AFE-CNN adopts a light-weight CNN model to decode images with action feature enhanced, which ensures a much lower computational burden than the state-of-the-art methods. We evaluate the AFE-CNN on three benchmark skeleton-based action datasets: NTU RGB+D, NTU RGB+D 120, and UTKinect-Action3D, with extensive experimental results demonstrate our outstanding performance of AFE-CNN.