Adults as Augmentations for Children in Facial Emotion Recognition with Contrastive Learning

TL;DR

This paper demonstrates that incorporating adult facial expression data as augmentations in contrastive learning significantly improves facial emotion recognition accuracy in children, addressing data scarcity issues in pediatric healthcare applications.

Contribution

It introduces a novel training scheme that explicitly uses adult data as augmentations for children's facial images in contrastive learning, enhancing emotion recognition performance.

Findings

Supervised contrastive learning with adult augmentations outperforms other methods by 2-3% in accuracy.

Adult data can effectively serve as meaningful augmentations for pediatric emotion recognition.

The approach opens new avenues for applying contrastive learning in pediatric healthcare contexts.

Abstract

Emotion recognition in children can help the early identification of, and intervention on, psychological complications that arise in stressful situations such as cancer treatment. Though deep learning models are increasingly being adopted, data scarcity is often an issue in pediatric medicine, including for facial emotion recognition in children. In this paper, we study the application of data augmentation-based contrastive learning to overcome data scarcity in facial emotion recognition for children. We explore the idea of ignoring generational gaps, by adding abundantly available adult data to pediatric data, to learn better representations. We investigate different ways by which adult facial expression images can be used alongside those of children. In particular, we propose to explicitly incorporate within each mini-batch adult images as augmentations for children's. Out of $84$ combinations of learning approaches and training set sizes, we find that supervised contrastive learning with the proposed training scheme performs best, reaching a test accuracy that typically surpasses the one of the second-best approach by 2% to 3%. Our results indicate that adult data can be considered to be a meaningful augmentation of pediatric data for the recognition of emotional facial expression in children, and open up the possibility for other applications of contrastive learning to improve pediatric care by complementing data of children with that of adults.