Identifying Emotions from Walking using Affective and Deep Features

TL;DR

This paper introduces a novel gait-based model that combines deep and affective features to classify perceived emotions from walking videos, achieving over 80% accuracy and also predicting valence and arousal.

Contribution

It is the first to use gait features with deep learning and affective cues for emotion recognition from walking videos, and introduces the EWalk dataset.

Findings

Achieved 80.07% accuracy in emotion classification

Successfully predicted perceived valence and arousal from gait

First gait-based model for emotion recognition from walking videos

Abstract

We present a new data-driven model and algorithm to identify the perceived emotions of individuals based on their walking styles. Given an RGB video of an individual walking, we extract his/her walking gait in the form of a series of 3D poses. Our goal is to exploit the gait features to classify the emotional state of the human into one of four emotions: happy, sad, angry, or neutral. Our perceived emotion recognition approach uses deep features learned via LSTM on labeled emotion datasets. Furthermore, we combine these features with affective features computed from gaits using posture and movement cues. These features are classified using a Random Forest Classifier. We show that our mapping between the combined feature space and the perceived emotional state provides 80.07% accuracy in identifying the perceived emotions. In addition to classifying discrete categories of emotions, our algorithm also predicts the values of perceived valence and arousal from gaits. We also present an EWalk (Emotion Walk) dataset that consists of videos of walking individuals with gaits and labeled emotions. To the best of our knowledge, this is the first gait-based model to identify perceived emotions from videos of walking individuals.