Emotion Recognition from Skeleton Data: A Comprehensive Survey

TL;DR

This survey reviews skeleton-based emotion recognition techniques, datasets, and applications, highlighting recent advances, technical paradigms, and future challenges in body movement-based emotional analysis.

Contribution

It provides a comprehensive taxonomy of methods, benchmarks, and applications, integrating recent developments and proposing a unified framework for skeleton-based emotion recognition.

Findings

Four primary technical paradigms identified: Traditional, Feat2Net, FeatFusionNet, End2EndNet

Benchmarking results across common datasets show varying performance levels

Extended applications include mental health assessment for depression and autism

Abstract

Emotion recognition through body movements has emerged as a compelling and privacy-preserving alternative to traditional methods that rely on facial expressions or physiological signals. Recent advancements in 3D skeleton acquisition technologies and pose estimation algorithms have significantly enhanced the feasibility of emotion recognition based on full-body motion. This survey provides a comprehensive and systematic review of skeleton-based emotion recognition techniques. First, we introduce psychological models of emotion and examine the relationship between bodily movements and emotional expression. Next, we summarize publicly available datasets, highlighting the differences in data acquisition methods and emotion labeling strategies. We then categorize existing methods into posture-based and gait-based approaches, analyzing them from both data-driven and technical perspectives. In particular, we propose a unified taxonomy that encompasses four primary technical paradigms: Traditional approaches, Feat2Net, FeatFusionNet, and End2EndNet. Representative works within each category are reviewed and compared, with benchmarking results across commonly used datasets. Finally, we explore the extended applications of emotion recognition in mental health assessment, such as detecting depression and autism, and discuss the open challenges and future research directions in this rapidly evolving field.