BYEL : Bootstrap Your Emotion Latent

TL;DR

BYEL is a novel framework that leverages synthetic images and self-supervised learning techniques to improve emotion recognition performance without requiring real image annotations.

Contribution

It introduces a new approach combining BYOL with emotion-specific modules to train solely on synthetic data for emotion analysis.

Findings

BYEL outperforms baseline by 2.8% in macro F1 score.

The framework effectively learns emotion representations from synthetic images.

It reduces dependency on real annotated data in emotion recognition tasks.

Abstract

With the improved performance of deep learning, the number of studies trying to apply deep learning to human emotion analysis is increasing rapidly. But even with this trend going on, it is still difficult to obtain high-quality images and annotations. For this reason, the Learning from Synthetic Data (LSD) Challenge, which learns from synthetic images and infers from real images, is one of the most interesting areas. In general, Domain Adaptation methods are widely used to address LSD challenges, but there is a limitation that target domains (real images) are still needed. Focusing on these limitations, we propose a framework Bootstrap Your Emotion Latent (BYEL), which uses only synthetic images in training. BYEL is implemented by adding Emotion Classifiers and Emotion Vector Subtraction to the BYOL framework that performs well in Self-Supervised Representation Learning. We train our framework using synthetic images generated from the Aff-wild2 dataset and evaluate it using real images from the Aff-wild2 dataset. The result shows that our framework (0.3084) performs 2.8% higher than the baseline (0.3) on the macro F1 score metric.