Towards Synthetic Data Generation for Improved Pain Recognition in Videos under Patient Constraints

TL;DR

This paper presents a synthetic data generation pipeline for pain recognition in videos, addressing ethical and data scarcity issues, and demonstrating improved model performance with synthetic and limited real data.

Contribution

It introduces a novel facial synthesis pipeline that creates diverse, realistic pain expressions for training, enhancing privacy and scalability in pain recognition models.

Findings

Synthetic data improves pain recognition accuracy.

Models trained on synthetic plus limited real data outperform those with only real data.

The approach ensures privacy by anonymizing identities.

Abstract

Recognizing pain in video is crucial for improving patient-computer interaction systems, yet traditional data collection in this domain raises significant ethical and logistical challenges. This study introduces a novel approach that leverages synthetic data to enhance video-based pain recognition models, providing an ethical and scalable alternative. We present a pipeline that synthesizes realistic 3D facial models by capturing nuanced facial movements from a small participant pool, and mapping these onto diverse synthetic avatars. This process generates 8,600 synthetic faces, accurately reflecting genuine pain expressions from varied angles and perspectives. Utilizing advanced facial capture techniques, and leveraging public datasets like CelebV-HQ and FFHQ-UV for demographic diversity, our new synthetic dataset significantly enhances model training while ensuring privacy by anonymizing identities through facial replacements. Experimental results demonstrate that models trained on combinations of synthetic data paired with a small amount of real participants achieve superior performance in pain recognition, effectively bridging the gap between synthetic simulations and real-world applications. Our approach addresses data scarcity and ethical concerns, offering a new solution for pain detection and opening new avenues for research in privacy-preserving dataset generation. All resources are publicly available to encourage further innovation in this field.