Progressive Spatio-Temporal Bilinear Network with Monte Carlo Dropout for Landmark-based Facial Expression Recognition with Uncertainty Estimation

TL;DR

This paper introduces a compact spatio-temporal bilinear neural network with Monte Carlo Dropout for facial expression recognition, effectively capturing landmark motion and uncertainty with less complexity than existing methods.

Contribution

It proposes a novel optimized network topology utilizing localized landmark features, a spatio-temporal bilinear layer, and uncertainty estimation for real-time facial expression recognition.

Findings

Comparable performance to state-of-the-art video methods

Less model complexity and faster inference

Effective uncertainty quantification for uncertain cases

Abstract

Deep neural networks have been widely used for feature learning in facial expression recognition systems. However, small datasets and large intra-class variability can lead to overfitting. In this paper, we propose a method which learns an optimized compact network topology for real-time facial expression recognition utilizing localized facial landmark features. Our method employs a spatio-temporal bilinear layer as backbone to capture the motion of facial landmarks during the execution of a facial expression effectively. Besides, it takes advantage of Monte Carlo Dropout to capture the model's uncertainty which is of great importance to analyze and treat uncertain cases. The performance of our method is evaluated on three widely used datasets and it is comparable to that of video-based state-of-the-art methods while it has much less complexity.