Skeletal Movement to Color Map: A Novel Representation for 3D Action Recognition with Inception Residual Networks

TL;DR

This paper introduces a new skeleton-based representation called SPMF for 3D action recognition, combined with optimized Inception Residual D-CNNs, achieving superior accuracy with less computation on challenging datasets.

Contribution

The paper presents a novel SPMF representation capturing postures and motions, and designs specialized Inception Residual D-CNNs for effective action recognition.

Findings

Outperforms state-of-the-art methods on MSR Action3D and NTU-RGB+D datasets.

Requires less computation than existing approaches.

Successfully captures complex human actions through the new representation.

Abstract

We propose a novel skeleton-based representation for 3D action recognition in videos using Deep Convolutional Neural Networks (D-CNNs). Two key issues have been addressed: First, how to construct a robust representation that easily captures the spatial-temporal evolutions of motions from skeleton sequences. Second, how to design D-CNNs capable of learning discriminative features from the new representation in a effective manner. To address these tasks, a skeletonbased representation, namely, SPMF (Skeleton Pose-Motion Feature) is proposed. The SPMFs are built from two of the most important properties of a human action: postures and their motions. Therefore, they are able to effectively represent complex actions. For learning and recognition tasks, we design and optimize new D-CNNs based on the idea of Inception Residual networks to predict actions from SPMFs. Our method is evaluated on two challenging datasets including MSR Action3D and NTU-RGB+D. Experimental results indicated that the proposed method surpasses state-of-the-art methods whilst requiring less computation.