Morphology-Independent Facial Expression Imitation for Human-Face Robots

TL;DR

This paper introduces a morphology-independent facial expression imitation method for human-face robots, which disentangles expression semantics from facial morphology to improve realism and robustness in expression imitation.

Contribution

It proposes a novel self-supervised decoupling approach for expression and morphology, enhancing expression imitation accuracy on human-face robots.

Findings

Effective reproduction of diverse human expressions

Robustness against facial morphology variations

Successful implementation on a custom human-face robot

Abstract

Accurate facial expression imitation on human-face robots is crucial for achieving natural human-robot interaction. Most existing methods have achieved photorealistic expression imitation through mapping 2D facial landmarks to a robot's actuator commands. Their imitation of landmark trajectories is susceptible to interference from facial morphology, which would lead to a performance drop. In this paper, we propose a morphology-independent expression imitation method that decouples expressions from facial morphology to eliminate morphological influence and produce more realistic expressions for human-face robots. Specifically, we construct an expression decoupling module to learn expression semantics by disentangling the expression representation from the morphology representation in a self-supervised manner. We devise an expression transfer module to map the representations to the robot's actuator commands through a learning objective of perceiving expression errors, producing accurate facial expressions based on the learned expression semantics. To support experimental validation, a custom-designed and highly expressive human-face robot, namely Pengrui, is developed to serve as an experimental platform for realistic expression imitation. Extensive experiments demonstrate that our method enables the human-face robot to reproduce a wide range of human-like expressions effectively. All code and implementation details of the robot will be released.