Data Augmentation for 3DMM-based Arousal-Valence Prediction for HRI

TL;DR

This paper introduces a data augmentation technique using 3D morphable models to enhance arousal-valence prediction accuracy in human-robot interaction scenarios, addressing challenges of diverse expressions and conditions.

Contribution

The study presents a novel data augmentation method for 3DMM-based AV prediction, improving model robustness and accuracy in HRI environments.

Findings

Improved AV prediction accuracy to 0.793 on SEWA dataset.

Enhanced robustness of AV models in real-time HRI settings.

Synthetic data generation benefits underrepresented AV values.

Abstract

Humans use multiple communication channels to interact with each other. For instance, body gestures or facial expressions are commonly used to convey an intent. The use of such non-verbal cues has motivated the development of prediction models. One such approach is predicting arousal and valence (AV) from facial expressions. However, making these models accurate for human-robot interaction (HRI) settings is challenging as it requires handling multiple subjects, challenging conditions, and a wide range of facial expressions. In this paper, we propose a data augmentation (DA) technique to improve the performance of AV predictors using 3D morphable models (3DMM). We then utilize this approach in an HRI setting with a mediator robot and a group of three humans. Our augmentation method creates synthetic sequences for underrepresented values in the AV space of the SEWA dataset, which is the most comprehensive dataset with continuous AV labels. Results show that using our DA method improves the accuracy and robustness of AV prediction in real-time applications. The accuracy of our models on the SEWA dataset is 0.793 for arousal and valence.