Towards Semi-Supervised Deep Facial Expression Recognition with An Adaptive Confidence Margin

TL;DR

This paper introduces Ada-CM, an adaptive confidence margin for semi-supervised deep facial expression recognition, enabling full utilization of unlabeled data and achieving state-of-the-art results.

Contribution

We propose an adaptive confidence margin that partitions unlabeled data dynamically, improving semi-supervised facial expression recognition performance.

Findings

Achieves state-of-the-art results on four datasets.

Surpasses fully-supervised baselines in semi-supervised setting.

Effective use of all unlabeled data enhances recognition accuracy.

Abstract

Only parts of unlabeled data are selected to train models for most semi-supervised learning methods, whose confidence scores are usually higher than the pre-defined threshold (i.e., the confidence margin). We argue that the recognition performance should be further improved by making full use of all unlabeled data. In this paper, we learn an Adaptive Confidence Margin (Ada-CM) to fully leverage all unlabeled data for semi-supervised deep facial expression recognition. All unlabeled samples are partitioned into two subsets by comparing their confidence scores with the adaptively learned confidence margin at each training epoch: (1) subset I including samples whose confidence scores are no lower than the margin; (2) subset II including samples whose confidence scores are lower than the margin. For samples in subset I, we constrain their predictions to match pseudo labels. Meanwhile, samples in subset II participate in the feature-level contrastive objective to learn effective facial expression features. We extensively evaluate Ada-CM on four challenging datasets, showing that our method achieves state-of-the-art performance, especially surpassing fully-supervised baselines in a semi-supervised manner. Ablation study further proves the effectiveness of our method. The source code is available at https://github.com/hangyu94/Ada-CM.