Co-Speech Gesture and Facial Expression Generation for Non-Photorealistic 3D Characters

TL;DR

This paper introduces methods for generating expressive bodily and facial cues for non-photorealistic 3D characters, enhancing emotional communication in anime-style avatars using comic-derived data.

Contribution

It presents novel techniques for emotion expression in non-photorealistic characters, leveraging comic and dialogue data, addressing a gap in existing photorealistic-focused research.

Findings

User study shows significant improvement in expressiveness.

Effective use of comic and dialogue data for emotion modeling.

Enhanced emotional communication in non-photorealistic characters.

Abstract

With the advancement of conversational AI, research on bodily expressions, including gestures and facial expressions, has also progressed. However, many existing studies focus on photorealistic avatars, making them unsuitable for non-photorealistic characters, such as those found in anime. This study proposes methods for expressing emotions, including exaggerated expressions unique to non-photorealistic characters, by utilizing expression data extracted from comics and dialogue-specific semantic gestures. A user study demonstrated significant improvements across multiple aspects when compared to existing research.