Active Learning with Contrastive Pre-training for Facial Expression Recognition

TL;DR

This paper explores active learning for facial expression recognition, identifying challenges with existing methods and proposing contrastive self-supervised pre-training to improve sample selection, resulting in significant performance gains.

Contribution

It introduces a contrastive pre-training approach to enhance active learning effectiveness in FER, addressing the cold start problem and improving sample selection.

Findings

Contrastive pre-training improves active learning performance by up to 9.2%.

Existing active learning methods struggle with the cold start problem in FER.

Proposed two-step approach outperforms baseline methods significantly.

Abstract

Deep learning has played a significant role in the success of facial expression recognition (FER), thanks to large models and vast amounts of labelled data. However, obtaining labelled data requires a tremendous amount of human effort, time, and financial resources. Even though some prior works have focused on reducing the need for large amounts of labelled data using different unsupervised methods, another promising approach called active learning is barely explored in the context of FER. This approach involves selecting and labelling the most representative samples from an unlabelled set to make the best use of a limited 'labelling budget'. In this paper, we implement and study 8 recent active learning methods on three public FER datasets, FER13, RAF-DB, and KDEF. Our findings show that existing active learning methods do not perform well in the context of FER, likely suffering from a phenomenon called 'Cold Start', which occurs when the initial set of labelled samples is not well representative of the entire dataset. To address this issue, we propose contrastive self-supervised pre-training, which first learns the underlying representations based on the entire unlabelled dataset. We then follow this with the active learning methods and observe that our 2-step approach shows up to 9.2% improvement over random sampling and up to 6.7% improvement over the best existing active learning baseline without the pre-training. We will make the code for this study public upon publication at: github.com/ShuvenduRoy/ActiveFER.