Facial Expression Generation Aligned with Human Preference for Natural Dyadic Interaction

TL;DR

This paper introduces a novel facial expression generation method that uses human feedback and reinforcement learning to produce natural, contextually appropriate expressions for dyadic interactions, improving alignment with human preferences.

Contribution

It presents a new approach that incorporates human feedback into facial expression generation through a closed feedback loop and reinforcement learning, enabling more natural and socially aligned expressions.

Findings

Effective alignment with human preferences demonstrated

Superior performance on benchmark datasets

Dynamic response to conversational cues achieved

Abstract

Achieving natural dyadic interaction requires generating facial expressions that are emotionally appropriate and socially aligned with human preference. Human feedback offers a compelling mechanism to guide such alignment, yet how to effectively incorporate this feedback into facial expression generation remains underexplored. In this paper, we propose a facial expression generation method aligned with human preference by leveraging human feedback to produce contextually and emotionally appropriate expressions for natural dyadic interaction. A key to our method is framing the generation of identity-independent facial expressions as an action learning process, allowing human feedback to assess their validity free from visual or identity bias. We establish a closed feedback loop in which listener expressions dynamically respond to evolving conversational cues of the speaker. Concretely, we train a vision-language-action model via supervised fine-tuning to map the speaker's multimodal signals into controllable low-dimensional expression representations of a 3D morphable model. We further introduce a human-feedback reinforcement learning strategy that integrates the imitation of high-quality expression response with critic-guided optimization. Experiments on two benchmarks demonstrate that our method effectively aligns facial expressions with human preference and achieves superior performance.