Expressive Gaussian Human Avatars from Monocular RGB Video

TL;DR

This paper introduces EVA, a novel framework for creating highly expressive 3D human avatars from monocular RGB videos, emphasizing fine-grained facial and hand details through improved alignment, adaptive density control, and confidence-guided learning.

Contribution

EVA advances avatar expressiveness by integrating a plug-and-play alignment module, adaptive density control, and a confidence prediction mechanism for detailed 3D Gaussian modeling.

Findings

Outperforms existing methods in capturing fine-grained facial and hand details.

Improves alignment accuracy of SMPL-X models in wild video conditions.

Demonstrates superior qualitative and quantitative results on benchmark datasets.

Abstract

Nuanced expressiveness, particularly through fine-grained hand and facial expressions, is pivotal for enhancing the realism and vitality of digital human representations. In this work, we focus on investigating the expressiveness of human avatars when learned from monocular RGB video; a setting that introduces new challenges in capturing and animating fine-grained details. To this end, we introduce EVA, a drivable human model that meticulously sculpts fine details based on 3D Gaussians and SMPL-X, an expressive parametric human model. Focused on enhancing expressiveness, our work makes three key contributions. First, we highlight the critical importance of aligning the SMPL-X model with RGB frames for effective avatar learning. Recognizing the limitations of current SMPL-X prediction methods for in-the-wild videos, we introduce a plug-and-play module that significantly ameliorates misalignment issues. Second, we propose a context-aware adaptive density control strategy, which is adaptively adjusting the gradient thresholds to accommodate the varied granularity across body parts. Last but not least, we develop a feedback mechanism that predicts per-pixel confidence to better guide the learning of 3D Gaussians. Extensive experiments on two benchmarks demonstrate the superiority of our framework both quantitatively and qualitatively, especially on the fine-grained hand and facial details. See the project website at \url{https://evahuman.github.io}