Learning Contrastive Feature Representations for Facial Action Unit Detection

TL;DR

This paper proposes a contrastive learning framework for facial action unit detection that combines self-supervised and supervised signals, addressing class imbalance and noisy labels to improve detection accuracy across multiple benchmarks.

Contribution

The novel contrastive learning approach integrates self-supervised signals with supervised AU detection, employing re-weighting and sampling techniques to handle class imbalance and noisy labels.

Findings

Outperforms state-of-the-art AU detection methods on five benchmark datasets.

Effectively mitigates the impact of noisy labels through a specialized sampling technique.

Addresses class imbalance with a negative sample re-weighting strategy.

Abstract

For the Facial Action Unit (AU) detection task, accurately capturing the subtle facial differences between distinct AUs is essential for reliable detection. Additionally, AU detection faces challenges from class imbalance and the presence of noisy or false labels, which undermine detection accuracy. In this paper, we introduce a novel contrastive learning framework aimed for AU detection that incorporates both self-supervised and supervised signals, thereby enhancing the learning of discriminative features for accurate AU detection. To tackle the class imbalance issue, we employ a negative sample re-weighting strategy that adjusts the step size of updating parameters for minority and majority class samples. Moreover, to address the challenges posed by noisy and false AU labels, we employ a sampling technique that encompasses three distinct types of positive sample pairs. This enables us to inject self-supervised signals into the supervised signal, effectively mitigating the adverse effects of noisy labels. Our experimental assessments, conducted on five widely-utilized benchmark datasets (BP4D, DISFA, BP4D+, GFT and Aff-Wild2), underscore the superior performance of our approach compared to state-of-the-art methods of AU detection. Our code is available at https://github.com/Ziqiao-Shang/AUNCE.