HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting

TL;DR

HeadStudio introduces a novel 3D Gaussian splatting framework that creates realistic, animatable head avatars from text prompts, enabling high-quality, real-time rendering and smooth animation driven by speech or video.

Contribution

It presents a new method combining 3D Gaussian splatting with semantic animation for high-quality, text-driven, animatable head avatars, improving upon prior static and less detailed models.

Findings

Generates realistic avatars from text prompts.

Achieves real-time rendering at 40 fps with high resolution.

Supports smooth animation driven by speech and video.

Abstract

Creating digital avatars from textual prompts has long been a desirable yet challenging task. Despite the promising results achieved with 2D diffusion priors, current methods struggle to create high-quality and consistent animated avatars efficiently. Previous animatable head models like FLAME have difficulty in accurately representing detailed texture and geometry. Additionally, high-quality 3D static representations face challenges in semantically driving with dynamic priors. In this paper, we introduce \textbf{HeadStudio}, a novel framework that utilizes 3D Gaussian splatting to generate realistic and animatable avatars from text prompts. Firstly, we associate 3D Gaussians with animatable head prior model, facilitating semantic animation on high-quality 3D representations. To ensure consistent animation, we further enhance the optimization from initialization, distillation, and regularization to jointly learn the shape, texture, and animation. Extensive experiments demonstrate the efficacy of HeadStudio in generating animatable avatars from textual prompts, exhibiting appealing appearances. The avatars are capable of rendering high-quality real-time ($\geq 40$ fps) novel views at a resolution of 1024. Moreover, These avatars can be smoothly driven by real-world speech and video. We hope that HeadStudio can enhance digital avatar creation and gain popularity in the community. Code is at: https://github.com/ZhenglinZhou/HeadStudio.