Navigating Label Ambiguity for Facial Expression Recognition in the Wild

TL;DR

This paper introduces Navigating Label Ambiguity (NLA), a novel framework for facial expression recognition that effectively handles label ambiguity, noise, and class imbalance, leading to improved accuracy in real-world conditions.

Contribution

The paper presents the first framework to simultaneously address label ambiguity, noise, and class imbalance in facial expression recognition.

Findings

NLA outperforms existing methods in overall accuracy.

NLA demonstrates robustness against noisy samples.

NLA effectively improves mean accuracy across classes.

Abstract

Facial expression recognition (FER) remains a challenging task due to label ambiguity caused by the subjective nature of facial expressions and noisy samples. Additionally, class imbalance, which is common in real-world datasets, further complicates FER. Although many studies have shown impressive improvements, they typically address only one of these issues, leading to suboptimal results. To tackle both challenges simultaneously, we propose a novel framework called Navigating Label Ambiguity (NLA), which is robust under real-world conditions. The motivation behind NLA is that dynamically estimating and emphasizing ambiguous samples at each iteration helps mitigate noise and class imbalance by reducing the model's bias toward majority classes. To achieve this, NLA consists of two main components: Noise-aware Adaptive Weighting (NAW) and consistency regularization. Specifically, NAW adaptively assigns higher importance to ambiguous samples and lower importance to noisy ones, based on the correlation between the intermediate prediction scores for the ground truth and the nearest negative. Moreover, we incorporate a regularization term to ensure consistent latent distributions. Consequently, NLA enables the model to progressively focus on more challenging ambiguous samples, which primarily belong to the minority class, in the later stages of training. Extensive experiments demonstrate that NLA outperforms existing methods in both overall and mean accuracy, confirming its robustness against noise and class imbalance. To the best of our knowledge, this is the first framework to address both problems simultaneously.