Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Setting

TL;DR

This paper presents a Latent Behavior Diffusion Model that synthesizes diverse, contextually relevant facial reactions in dyadic conversations, improving naturalness in human-like interaction simulations.

Contribution

The paper introduces a novel diffusion-based generative model with a context-aware autoencoder for realistic reaction synthesis in dyadic settings.

Findings

Outperforms existing methods in reaction synthesis accuracy

Generates diverse facial reactions reflecting subtle conversational cues

Effective in capturing emotional nuances in reactions

Abstract

The dyadic reaction generation task involves synthesizing responsive facial reactions that align closely with the behaviors of a conversational partner, enhancing the naturalness and effectiveness of human-like interaction simulations. This paper introduces a novel approach, the Latent Behavior Diffusion Model, comprising a context-aware autoencoder and a diffusion-based conditional generator that addresses the challenge of generating diverse and contextually relevant facial reactions from input speaker behaviors. The autoencoder compresses high-dimensional input features, capturing dynamic patterns in listener reactions while condensing complex input data into a concise latent representation, facilitating more expressive and contextually appropriate reaction synthesis. The diffusion-based conditional generator operates on the latent space generated by the autoencoder to predict realistic facial reactions in a non-autoregressive manner. This approach allows for generating diverse facial reactions that reflect subtle variations in conversational cues and emotional states. Experimental results demonstrate the effectiveness of our approach in achieving superior performance in dyadic reaction synthesis tasks compared to existing methods.