CAGE: Circumplex Affect Guided Expression Inference

TL;DR

This paper introduces CAGE, a lightweight facial expression inference model that leverages the circumplex model of affect, improving accuracy by incorporating continuous valence and arousal labels alongside discrete categories.

Contribution

It presents a novel MaxViT-based model that effectively combines discrete and continuous emotion representations for expression inference, outperforming current state-of-the-art methods.

Findings

The model achieves a 7% lower RMSE on AffectNet.

Considering valence and arousal improves expression inference.

The approach outperforms existing models on key metrics.

Abstract

Understanding emotions and expressions is a task of interest across multiple disciplines, especially for improving user experiences. Contrary to the common perception, it has been shown that emotions are not discrete entities but instead exist along a continuum. People understand discrete emotions differently due to a variety of factors, including cultural background, individual experiences, and cognitive biases. Therefore, most approaches to expression understanding, particularly those relying on discrete categories, are inherently biased. In this paper, we present a comparative in-depth analysis of two common datasets (AffectNet and EMOTIC) equipped with the components of the circumplex model of affect. Further, we propose a model for the prediction of facial expressions tailored for lightweight applications. Using a small-scaled MaxViT-based model architecture, we evaluate the impact of discrete expression category labels in training with the continuous valence and arousal labels. We show that considering valence and arousal in addition to discrete category labels helps to significantly improve expression inference. The proposed model outperforms the current state-of-the-art models on AffectNet, establishing it as the best-performing model for inferring valence and arousal achieving a 7% lower RMSE. Training scripts and trained weights to reproduce our results can be found here: https://github.com/wagner-niklas/CAGE_expression_inference.