X2C: A Dataset Featuring Nuanced Facial Expressions for Realistic Humanoid Imitation

TL;DR

The paper introduces X2C, a large-scale dataset of humanoid facial expressions with annotations, and X2CNet, a framework for realistic imitation of these expressions, demonstrated on a physical robot.

Contribution

It provides a new high-diversity dataset and a novel imitation framework for humanoid facial expressions, advancing research in affective human-robot communication.

Findings

X2C dataset contains 100,000 annotated image-expression pairs.

X2CNet successfully learns expression-to-control value mapping.

Real-world robot demonstrations validate the approach.

Abstract

The ability to imitate realistic facial expressions is essential for humanoid robots engaged in affective human-robot communication. However, the lack of datasets containing diverse humanoid facial expressions with proper annotations hinders progress in realistic humanoid facial expression imitation. To address these challenges, we introduce X2C (Anything to Control), a dataset featuring nuanced facial expressions for realistic humanoid imitation. With X2C, we contribute: 1) a high-quality, high-diversity, large-scale dataset comprising 100,000 (image, control value) pairs. Each image depicts a humanoid robot displaying a diverse range of facial expressions, annotated with 30 control values representing the ground-truth expression configuration; 2) X2CNet, a novel human-to-humanoid facial expression imitation framework that learns the correspondence between nuanced humanoid expressions and their underlying control values from X2C. It enables facial expression imitation in the wild for different human performers, providing a baseline for the imitation task, showcasing the potential value of our dataset; 3) real-world demonstrations on a physical humanoid robot, highlighting its capability to advance realistic humanoid facial expression imitation. Code and Data: https://lipzh5.github.io/X2CNet/