3D Skeleton-Based Action Recognition: A Review

TL;DR

This review comprehensively analyzes 3D skeleton-based action recognition by decomposing the task into sub-tasks, emphasizing preprocessing, feature extraction, and recent advanced models, providing a structured roadmap for future research.

Contribution

It offers a task-oriented framework that covers fundamental sub-tasks, recent advancements, and benchmark datasets, addressing gaps in previous model-focused reviews.

Findings

Decomposition of the recognition task into sub-tasks like preprocessing and feature extraction.

Analysis of recent models including hybrid architectures, Mamba, LLMs, and generative models.

Overview of public datasets and evaluation benchmarks.

Abstract

With the inherent advantages of skeleton representation, 3D skeleton-based action recognition has become a prominent topic in the field of computer vision. However, previous reviews have predominantly adopted a model-oriented perspective, often neglecting the fundamental steps involved in skeleton-based action recognition. This oversight tends to ignore key components of skeleton-based action recognition beyond model design and has hindered deeper, more intrinsic understanding of the task. To bridge this gap, our review aims to address these limitations by presenting a comprehensive, task-oriented framework for understanding skeleton-based action recognition. We begin by decomposing the task into a series of sub-tasks, placing particular emphasis on preprocessing steps such as modality derivation and data augmentation. The subsequent discussion delves into critical sub-tasks, including feature extraction and spatio-temporal modeling techniques. Beyond foundational action recognition networks, recently advanced frameworks such as hybrid architectures, Mamba models, large language models (LLMs), and generative models have also been highlighted. Finally, a comprehensive overview of public 3D skeleton datasets is presented, accompanied by an analysis of state-of-the-art algorithms evaluated on these benchmarks. By integrating task-oriented discussions, comprehensive examinations of sub-tasks, and an emphasis on the latest advancements, our review provides a fundamental and accessible structured roadmap for understanding and advancing the field of 3D skeleton-based action recognition.