Skeleton based Activity Recognition by Fusing Part-wise Spatio-temporal and Attention Driven Residues

TL;DR

This paper introduces a novel skeleton-based 3D human action recognition framework that combines part-wise spatiotemporal features with attention-driven residues, achieving state-of-the-art accuracy on multiple datasets.

Contribution

It proposes a new part-wise spatiotemporal CNN architecture with attention-driven residues for improved skeleton-based action recognition.

Findings

Achieves highest top-1 accuracy on benchmark datasets

Demonstrates robustness across multiple challenging datasets

Highlights local skeleton features effectively

Abstract

There exist a wide range of intra class variations of the same actions and inter class similarity among the actions, at the same time, which makes the action recognition in videos very challenging. In this paper, we present a novel skeleton-based part-wise Spatiotemporal CNN RIAC Network-based 3D human action recognition framework to visualise the action dynamics in part wise manner and utilise each part for action recognition by applying weighted late fusion mechanism. Part wise skeleton based motion dynamics helps to highlight local features of the skeleton which is performed by partitioning the complete skeleton in five parts such as Head to Spine, Left Leg, Right Leg, Left Hand, Right Hand. The RIAFNet architecture is greatly inspired by the InceptionV4 architecture which unified the ResNet and Inception based Spatio-temporal feature representation concept and achieving the highest top-1 accuracy till date. To extract and learn salient features for action recognition, attention driven residues are used which enhance the performance of residual components for effective 3D skeleton-based Spatio-temporal action representation. The robustness of the proposed framework is evaluated by performing extensive experiments on three challenging datasets such as UT Kinect Action 3D, Florence 3D action Dataset, and MSR Daily Action3D datasets, which consistently demonstrate the superiority of our method