Synthetic Thermal and RGB Videos for Automatic Pain Assessment utilizing a Vision-MLP Architecture

TL;DR

This paper introduces a novel framework combining synthetic thermal videos generated by GANs with RGB data, using a Vision-MLP architecture for improved automatic pain assessment.

Contribution

It presents a new approach integrating synthetic thermal videos into pain recognition, demonstrating their effectiveness alongside RGB data.

Findings

Synthetic thermal videos improve pain detection accuracy.

Multimodal RGB and thermal data outperform unimodal approaches.

GAN-generated thermal videos are effective for pain assessment.

Abstract

Pain assessment is essential in developing optimal pain management protocols to alleviate suffering and prevent functional decline in patients. Consequently, reliable and accurate automatic pain assessment systems are essential for continuous and effective patient monitoring. This study presents synthetic thermal videos generated by Generative Adversarial Networks integrated into the pain recognition pipeline and evaluates their efficacy. A framework consisting of a Vision-MLP and a Transformer-based module is utilized, employing RGB and synthetic thermal videos in unimodal and multimodal settings. Experiments conducted on facial videos from the BioVid database demonstrate the effectiveness of synthetic thermal videos and underline the potential advantages of it.