Uncertain Facial Expression Recognition via Multi-task Assisted Correction

TL;DR

This paper introduces MTAC, a multi-task approach that improves facial expression recognition by addressing data uncertainty through auxiliary tasks, confidence estimation, and relabeling strategies, enhancing robustness and accuracy.

Contribution

The paper proposes a novel multi-task assisted correction method that effectively handles uncertain facial expressions by integrating auxiliary tasks and a relabeling strategy.

Findings

MTAC significantly improves recognition accuracy on RAF-DB, AffectNet, and AffWild2 datasets.

The method outperforms existing state-of-the-art approaches under uncertain conditions.

Experiments demonstrate robustness against synthetic and real annotation uncertainties.

Abstract

Deep models for facial expression recognition achieve high performance by training on large-scale labeled data. However, publicly available datasets contain uncertain facial expressions caused by ambiguous annotations or confusing emotions, which could severely decline the robustness. Previous studies usually follow the bias elimination method in general tasks without considering the uncertainty problem from the perspective of different corresponding sources. In this paper, we propose a novel method of multi-task assisted correction in addressing uncertain facial expression recognition called MTAC. Specifically, a confidence estimation block and a weighted regularization module are applied to highlight solid samples and suppress uncertain samples in every batch. In addition, two auxiliary tasks, i.e., action unit detection and valence-arousal measurement, are introduced to learn semantic distributions from a data-driven AU graph and mitigate category imbalance based on latent dependencies between discrete and continuous emotions, respectively. Moreover, a re-labeling strategy guided by feature-level similarity constraint further generates new labels for identified uncertain samples to promote model learning. The proposed method can flexibly combine with existing frameworks in a fully-supervised or weakly-supervised manner. Experiments on RAF-DB, AffectNet, and AffWild2 datasets demonstrate that the MTAC obtains substantial improvements over baselines when facing synthetic and real uncertainties and outperforms the state-of-the-art methods.