EVA: Expressive Virtual Avatars from Multi-view Videos

TL;DR

EVA introduces a fully controllable, expressive human avatar framework from multi-view videos, enabling independent manipulation of facial expressions, body movements, and gestures with high realism and real-time rendering.

Contribution

The paper presents a novel two-layer avatar model and an expressive template tracking algorithm, improving disentanglement and control over human avatar features compared to prior methods.

Findings

EVA achieves superior rendering quality and expressiveness.

It enables independent control of facial expressions, body, and hand gestures.

Experimental results validate its effectiveness over state-of-the-art approaches.

Abstract

With recent advancements in neural rendering and motion capture algorithms, remarkable progress has been made in photorealistic human avatar modeling, unlocking immense potential for applications in virtual reality, augmented reality, remote communication, and industries such as gaming, film, and medicine. However, existing methods fail to provide complete, faithful, and expressive control over human avatars due to their entangled representation of facial expressions and body movements. In this work, we introduce Expressive Virtual Avatars (EVA), an actor-specific, fully controllable, and expressive human avatar framework that achieves high-fidelity, lifelike renderings in real time while enabling independent control of facial expressions, body movements, and hand gestures. Specifically, our approach designs the human avatar as a two-layer model: an expressive template geometry layer and a 3D Gaussian appearance layer. First, we present an expressive template tracking algorithm that leverages coarse-to-fine optimization to accurately recover body motions, facial expressions, and non-rigid deformation parameters from multi-view videos. Next, we propose a novel decoupled 3D Gaussian appearance model designed to effectively disentangle body and facial appearance. Unlike unified Gaussian estimation approaches, our method employs two specialized and independent modules to model the body and face separately. Experimental results demonstrate that EVA surpasses state-of-the-art methods in terms of rendering quality and expressiveness, validating its effectiveness in creating full-body avatars. This work represents a significant advancement towards fully drivable digital human models, enabling the creation of lifelike digital avatars that faithfully replicate human geometry and appearance.