GVA: Reconstructing Vivid 3D Gaussian Avatars from Monocular Videos

TL;DR

This paper introduces a new method for creating vivid 3D Gaussian avatars from monocular videos, improving pose accuracy and surface alignment to achieve high-fidelity, photo-realistic reconstructions with fine control.

Contribution

It presents a pose refinement technique and a surface-guided re-initialization method to enhance 3D Gaussian avatar quality from monocular videos.

Findings

Achieves state-of-the-art photo-realistic view synthesis.

Provides fine-grained control over human body and hand pose.

Demonstrates high-fidelity and vivid 3D avatar reconstructions.

Abstract

In this paper, we present a novel method that facilitates the creation of vivid 3D Gaussian avatars from monocular video inputs (GVA). Our innovation lies in addressing the intricate challenges of delivering high-fidelity human body reconstructions and aligning 3D Gaussians with human skin surfaces accurately. The key contributions of this paper are twofold. Firstly, we introduce a pose refinement technique to improve hand and foot pose accuracy by aligning normal maps and silhouettes. Precise pose is crucial for correct shape and appearance reconstruction. Secondly, we address the problems of unbalanced aggregation and initialization bias that previously diminished the quality of 3D Gaussian avatars, through a novel surface-guided re-initialization method that ensures accurate alignment of 3D Gaussian points with avatar surfaces. Experimental results demonstrate that our proposed method achieves high-fidelity and vivid 3D Gaussian avatar reconstruction. Extensive experimental analyses validate the performance qualitatively and quantitatively, demonstrating that it achieves state-of-the-art performance in photo-realistic novel view synthesis while offering fine-grained control over the human body and hand pose. Project page: https://3d-aigc.github.io/GVA/.