LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animation

TL;DR

LiftAvatar is a novel method that completes sparse monocular observations in kinematic space to enable high-fidelity, expression-controlled 3D avatar animation with improved realism and consistency.

Contribution

It introduces a kinematic-space completion framework with multi-granularity control and multi-reference conditioning for enhanced 3D avatar animation.

Findings

Boosts animation quality and metrics of state-of-the-art methods

Enables expressive and consistent avatar animations from monocular videos

Addresses artifacts caused by sparse kinematic cues

Abstract

We present LiftAvatar, a new paradigm that completes sparse monocular observations in kinematic space (e.g., facial expressions and head pose) and uses the completed signals to drive high-fidelity avatar animation. LiftAvatar is a fine-grained, expression-controllable large-scale video diffusion Transformer that synthesizes high-quality, temporally coherent expression sequences conditioned on single or multiple reference images. The key idea is to lift incomplete input data into a richer kinematic representation, thereby strengthening both reconstruction and animation in downstream 3D avatar pipelines. To this end, we introduce (i) a multi-granularity expression control scheme that combines shading maps with expression coefficients for precise and stable driving, and (ii) a multi-reference conditioning mechanism that aggregates complementary cues from multiple frames, enabling strong 3D consistency and controllability. As a plug-and-play enhancer, LiftAvatar directly addresses the limited expressiveness and reconstruction artifacts of 3D Gaussian Splatting-based avatars caused by sparse kinematic cues in everyday monocular videos. By expanding incomplete observations into diverse pose-expression variations, LiftAvatar also enables effective prior distillation from large-scale video generative models into 3D pipelines, leading to substantial gains. Extensive experiments show that LiftAvatar consistently boosts animation quality and quantitative metrics of state-of-the-art 3D avatar methods, especially under extreme, unseen expressions.