Emotion Recognition with Spatial Attention and Temporal Softmax Pooling

TL;DR

This paper introduces a simplified yet effective method for video-based emotion recognition by combining a pre-trained CNN with spatial attention and temporal softmax pooling, outperforming complex models on the EmotiW dataset.

Contribution

It presents a novel approach that integrates spatial attention and temporal softmax pooling with a pre-trained CNN for improved emotion recognition accuracy.

Findings

Achieved higher accuracy than complex models on EmotiW dataset.

Demonstrated effectiveness of spatial attention in localizing key facial regions.

Showed that temporal softmax pooling improves frame selection for emotion recognition.

Abstract

Video-based emotion recognition is a challenging task because it requires to distinguish the small deformations of the human face that represent emotions, while being invariant to stronger visual differences due to different identities. State-of-the-art methods normally use complex deep learning models such as recurrent neural networks (RNNs, LSTMs, GRUs), convolutional neural networks (CNNs, C3D, residual networks) and their combination. In this paper, we propose a simpler approach that combines a CNN pre-trained on a public dataset of facial images with (1) a spatial attention mechanism, to localize the most important regions of the face for a given emotion, and (2) temporal softmax pooling, to select the most important frames of the given video. Results on the challenging EmotiW dataset show that this approach can achieve higher accuracy than more complex approaches.