Real-time Automatic Emotion Recognition from Body Gestures

TL;DR

This paper presents a real-time system for recognizing emotions from body gestures using 3D skeleton data, achieving recognition accuracy comparable to humans, and validated through motion capture and Kinect data.

Contribution

It introduces a novel approach combining postural and kinematic features with SVM classification for emotion recognition from body movements.

Findings

Recognition rate of 61.3%, close to human accuracy of 61.9%.

Effective on data from both optical motion capture and Kinect.

Two interactive games demonstrate practical application.

Abstract

Although psychological research indicates that bodily expressions convey important affective information, to date research in emotion recognition focused mainly on facial expression or voice analysis. In this paper we propose an approach to realtime automatic emotion recognition from body movements. A set of postural, kinematic, and geometrical features are extracted from sequences 3D skeletons and fed to a multi-class SVM classifier. The proposed method has been assessed on data acquired through two different systems: a professionalgrade optical motion capture system, and Microsoft Kinect. The system has been assessed on a "six emotions" recognition problem, and using a leave-one-subject-out cross validation strategy, reached an overall recognition rate of 61.3% which is very close to the recognition rate of 61.9% obtained by human observers. To provide further testing of the system, two games were developed, where one or two users have to interact to understand and express emotions with their body.