ASM: Adaptive Sample Mining for In-The-Wild Facial Expression Recognition

TL;DR

This paper introduces Adaptive Sample Mining (ASM), a novel method for facial expression recognition that dynamically handles ambiguous and noisy samples, improving robustness and accuracy across datasets.

Contribution

The paper proposes ASM, which adaptively identifies and processes ambiguous and noisy samples in FER datasets, enhancing model robustness and performance.

Findings

Outperforms state-of-the-art methods on synthetic noisy datasets

Effectively mines ambiguity and noise in real datasets

Improves discrimination ability through mutual learning and unsupervised strategies

Abstract

Given the similarity between facial expression categories, the presence of compound facial expressions, and the subjectivity of annotators, facial expression recognition (FER) datasets often suffer from ambiguity and noisy labels. Ambiguous expressions are challenging to differentiate from expressions with noisy labels, which hurt the robustness of FER models. Furthermore, the difficulty of recognition varies across different expression categories, rendering a uniform approach unfair for all expressions. In this paper, we introduce a novel approach called Adaptive Sample Mining (ASM) to dynamically address ambiguity and noise within each expression category. First, the Adaptive Threshold Learning module generates two thresholds, namely the clean and noisy thresholds, for each category. These thresholds are based on the mean class probabilities at each training epoch. Next, the Sample Mining module partitions the dataset into three subsets: clean, ambiguity, and noise, by comparing the sample confidence with the clean and noisy thresholds. Finally, the Tri-Regularization module employs a mutual learning strategy for the ambiguity subset to enhance discrimination ability, and an unsupervised learning strategy for the noise subset to mitigate the impact of noisy labels. Extensive experiments prove that our method can effectively mine both ambiguity and noise, and outperform SOTA methods on both synthetic noisy and original datasets. The supplement material is available at https://github.com/zzzzzzyang/ASM.