Text/Speech-Driven Full-Body Animation

TL;DR

This paper introduces a comprehensive system that synthesizes realistic full-body avatar animations driven by text and speech, combining learning-based facial animation with graph-based body motion for efficient, diverse, and synchronized results.

Contribution

The work presents a production-ready system integrating facial and body animation synthesis driven by text and speech, utilizing novel learning and graph-based methods for high-quality avatar animation.

Findings

Generated animations are realistic and diverse.

Animations are highly synchronized with input speech.

System operates efficiently and robustly.

Abstract

Due to the increasing demand in films and games, synthesizing 3D avatar animation has attracted much attention recently. In this work, we present a production-ready text/speech-driven full-body animation synthesis system. Given the text and corresponding speech, our system synthesizes face and body animations simultaneously, which are then skinned and rendered to obtain a video stream output. We adopt a learning-based approach for synthesizing facial animation and a graph-based approach to animate the body, which generates high-quality avatar animation efficiently and robustly. Our results demonstrate the generated avatar animations are realistic, diverse and highly text/speech-correlated.