Ensemble of Hankel Matrices for Face Emotion Recognition

TL;DR

This paper introduces an ensemble of Hankel matrices derived from appearance feature time series for face emotion recognition, achieving state-of-the-art accuracy on a public dataset.

Contribution

It proposes a novel approach using Hankel matrices of appearance features for emotion recognition, combining nearest neighbor and voting schemes.

Findings

Achieves state-of-the-art accuracy on a public dataset.

Demonstrates the effectiveness of Hankel matrix dynamics in emotion classification.

Shows promising results with the proposed representation.

Abstract

In this paper, a face emotion is considered as the result of the composition of multiple concurrent signals, each corresponding to the movements of a specific facial muscle. These concurrent signals are represented by means of a set of multi-scale appearance features that might be correlated with one or more concurrent signals. The extraction of these appearance features from a sequence of face images yields to a set of time series. This paper proposes to use the dynamics regulating each appearance feature time series to recognize among different face emotions. To this purpose, an ensemble of Hankel matrices corresponding to the extracted time series is used for emotion classification within a framework that combines nearest neighbor and a majority vote schema. Experimental results on a public available dataset shows that the adopted representation is promising and yields state-of-the-art accuracy in emotion classification.