ST-Gait++: Leveraging spatio-temporal convolutions for gait-based emotion recognition on videos

TL;DR

This paper introduces ST-Gait++, a deep learning framework using spatio-temporal graph convolutions to recognize emotions from gait videos, achieving higher accuracy and faster training convergence than existing methods.

Contribution

The work presents a novel skeleton-based emotion recognition model with spatial-temporal graph convolutions, outperforming state-of-the-art accuracy and convergence speed.

Findings

Approximately 5% accuracy improvement over previous methods

Faster convergence during training

Effective gait-based emotion recognition on E-Gait dataset

Abstract

Emotion recognition is relevant for human behaviour understanding, where facial expression and speech recognition have been widely explored by the computer vision community. Literature in the field of behavioural psychology indicates that gait, described as the way a person walks, is an additional indicator of emotions. In this work, we propose a deep framework for emotion recognition through the analysis of gait. More specifically, our model is composed of a sequence of spatial-temporal Graph Convolutional Networks that produce a robust skeleton-based representation for the task of emotion classification. We evaluate our proposed framework on the E-Gait dataset, composed of a total of 2177 samples. The results obtained represent an improvement of approximately 5% in accuracy compared to the state of the art. In addition, during training we observed a faster convergence of our model compared to the state-of-the-art methodologies.