Improvement in Facial Emotion Recognition using Synthetic Data Generated by Diffusion Model

TL;DR

This paper demonstrates that using synthetic facial emotion data generated by diffusion models significantly improves the accuracy of emotion recognition systems, especially in imbalanced datasets.

Contribution

It introduces a novel data augmentation approach using diffusion models to enhance FER performance with the ResEmoteNet model.

Findings

Achieved 96.47% accuracy on FER2013 dataset

Achieved 99.23% accuracy on RAF-DB dataset

Significant performance improvements over baseline models

Abstract

Facial Emotion Recognition (FER) plays a crucial role in computer vision, with significant applications in human-computer interaction, affective computing, and areas such as mental health monitoring and personalized learning environments. However, a major challenge in FER task is the class imbalance commonly found in available datasets, which can hinder both model performance and generalization. In this paper, we tackle the issue of data imbalance by incorporating synthetic data augmentation and leveraging the ResEmoteNet model to enhance the overall performance on facial emotion recognition task. We employed Stable Diffusion 2 and Stable Diffusion 3 Medium models to generate synthetic facial emotion data, augmenting the training sets of the FER2013 and RAF-DB benchmark datasets. Training ResEmoteNet with these augmented datasets resulted in substantial performance improvements, achieving accuracies of 96.47% on FER2013 and 99.23% on RAF-DB. These findings shows an absolute improvement of 16.68% in FER2013, 4.47% in RAF-DB and highlight the efficacy of synthetic data augmentation in strengthening FER models and underscore the potential of advanced generative models in FER research and applications. The source code for ResEmoteNet is available at https://github.com/ArnabKumarRoy02/ResEmoteNet