Emotion Recognition by Body Movement Representation on the Manifold of Symmetric Positive Definite Matrices

TL;DR

This paper introduces a novel approach for emotion recognition based on 3D full-body motion analysis using covariance descriptors on the manifold of Symmetric Positive Definite matrices, achieving classification performance comparable to human judgment.

Contribution

It proposes a new method that models whole-body motion on a Riemannian manifold for emotion recognition, extending beyond facial expression analysis.

Findings

Successfully classified five emotional states from 3D body motion sequences.

Achieved classification accuracy comparable to human force-choice tasks.

Utilized covariance descriptors and geometric means on the manifold for effective emotion recognition.

Abstract

Emotion recognition is attracting great interest for its potential application in a multitude of real-life situations. Much of the Computer Vision research in this field has focused on relating emotions to facial expressions, with investigations rarely including more than upper body. In this work, we propose a new scenario, for which emotional states are related to 3D dynamics of the whole body motion. To address the complexity of human body movement, we used covariance descriptors of the sequence of the 3D skeleton joints, and represented them in the non-linear Riemannian manifold of Symmetric Positive Definite matrices. In doing so, we exploited geodesic distances and geometric means on the manifold to perform emotion classification. Using sequences of spontaneous walking under the five primary emotional states, we report a method that succeeded in classifying the different emotions, with comparable performance to those observed in a human-based force-choice classification task.