Facial expression recognition via variational inference

TL;DR

This paper introduces POSTER-Var, a new framework for facial expression recognition that uses probabilistic modeling to better capture the complexity and uncertainty of real-world expressions.

Contribution

The novel contribution is the Variational Inference-based Classification Head (VICH), which models expression intensity distributions rather than deterministic classifications.

Findings

POSTER-Var achieves state-of-the-art performance on fine-grained expression recognition tasks.

The probabilistic latent space approach improves handling of compound and subtle expressions.

Layer embeddings and nonlinear transformations enhance hierarchical feature fusion.

Abstract

Facial expressions in the wild are rarely discrete; they often manifest as compound emotions or subtle variations that challenge the discriminative capabilities of conventional models. While psychological research suggests that expressions are often combinations of basic emotional units, most existing FER methods rely on deterministic point estimation, failing to model the intrinsic uncertainty and continuous nature of emotions. To address this, we propose POSTER-Var, a framework integrating a Variational Inference-based Classification Head (VICH). Unlike standard classifiers, VICH maps facial features into a probabilistic latent space via the reparameterization trick, enabling the model to learn the underlying distribution of expression intensities. Furthermore, we enhance feature representation by introducing layer embeddings and nonlinear transformations into the feature pyramid, facilitating the fusion of hierarchical semantic information. Extensive experiments on RAF-DB, AffectNet, and FER+ demonstrate that our method effectively handles fine-grained expression recognition, achieving state-of-the-art performance. The code has been open-sourced at: https://github.com/lg2578/poster-var.

Figures