ActionXPose: A Novel 2D Multi-view Pose-based Algorithm for Real-time Human Action Recognition

TL;DR

ActionXPose introduces a real-time, 2D pose-based algorithm for human action recognition that leverages pose data from RGB videos, demonstrating high accuracy and robustness across multiple datasets and challenging conditions.

Contribution

It is one of the first algorithms to utilize 2D human poses for HAR, combining LSTM and CNN for improved classification and generalization.

Findings

Achieves state-of-the-art results on i3DPost and KTH datasets.

Demonstrates robustness to camera movement and viewpoint changes.

Successfully trained on multiple datasets simultaneously.

Abstract

We present ActionXPose, a novel 2D pose-based algorithm for posture-level Human Action Recognition (HAR). The proposed approach exploits 2D human poses provided by OpenPose detector from RGB videos. ActionXPose aims to process poses data to be provided to a Long Short-Term Memory Neural Network and to a 1D Convolutional Neural Network, which solve the classification problem. ActionXPose is one of the first algorithms that exploits 2D human poses for HAR. The algorithm has real-time performance and it is robust to camera movings, subject proximity changes, viewpoint changes, subject appearance changes and provide high generalization degree. In fact, extensive simulations show that ActionXPose can be successfully trained using different datasets at once. State-of-the-art performance on popular datasets for posture-related HAR problems (i3DPost, KTH) are provided and results are compared with those obtained by other methods, including the selected ActionXPose baseline. Moreover, we also proposed two novel datasets called MPOSE and ISLD recorded in our Intelligent Sensing Lab, to show ActionXPose generalization performance.