MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar

TL;DR

MonoGaussianAvatar introduces a novel Gaussian point-based approach for animating photo-realistic head avatars from monocular videos, offering flexible topology, efficient deformation, and superior rendering performance.

Contribution

It proposes a new Gaussian point representation with a deformation field for head avatars, overcoming fixed topology and training challenges of prior methods.

Findings

Achieves state-of-the-art results in head avatar animation.

Demonstrates flexible topology and efficient deformation.

Provides high-quality rendering with controllable attributes.

Abstract

The ability to animate photo-realistic head avatars reconstructed from monocular portrait video sequences represents a crucial step in bridging the gap between the virtual and real worlds. Recent advancements in head avatar techniques, including explicit 3D morphable meshes (3DMM), point clouds, and neural implicit representation have been exploited for this ongoing research. However, 3DMM-based methods are constrained by their fixed topologies, point-based approaches suffer from a heavy training burden due to the extensive quantity of points involved, and the last ones suffer from limitations in deformation flexibility and rendering efficiency. In response to these challenges, we propose MonoGaussianAvatar (Monocular Gaussian Point-based Head Avatar), a novel approach that harnesses 3D Gaussian point representation coupled with a Gaussian deformation field to learn explicit head avatars from monocular portrait videos. We define our head avatars with Gaussian points characterized by adaptable shapes, enabling flexible topology. These points exhibit movement with a Gaussian deformation field in alignment with the target pose and expression of a person, facilitating efficient deformation. Additionally, the Gaussian points have controllable shape, size, color, and opacity combined with Gaussian splatting, allowing for efficient training and rendering. Experiments demonstrate the superior performance of our method, which achieves state-of-the-art results among previous methods.