3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting

TL;DR

This paper presents a fast and efficient method for creating animatable human avatars from monocular videos using 3D Gaussian Splatting, enabling real-time rendering and quick training, outperforming existing approaches in speed and quality.

Contribution

The authors introduce a novel approach combining 3D Gaussian Splatting with a non-rigid deformation network for rapid training and real-time rendering of animatable avatars from monocular videos.

Findings

Achieves real-time rendering at 50+ FPS.

Trains in approximately 30 minutes.

Outperforms state-of-the-art methods in speed and quality.

Abstract

We introduce an approach that creates animatable human avatars from monocular videos using 3D Gaussian Splatting (3DGS). Existing methods based on neural radiance fields (NeRFs) achieve high-quality novel-view/novel-pose image synthesis but often require days of training, and are extremely slow at inference time. Recently, the community has explored fast grid structures for efficient training of clothed avatars. Albeit being extremely fast at training, these methods can barely achieve an interactive rendering frame rate with around 15 FPS. In this paper, we use 3D Gaussian Splatting and learn a non-rigid deformation network to reconstruct animatable clothed human avatars that can be trained within 30 minutes and rendered at real-time frame rates (50+ FPS). Given the explicit nature of our representation, we further introduce as-isometric-as-possible regularizations on both the Gaussian mean vectors and the covariance matrices, enhancing the generalization of our model on highly articulated unseen poses. Experimental results show that our method achieves comparable and even better performance compared to state-of-the-art approaches on animatable avatar creation from a monocular input, while being 400x and 250x faster in training and inference, respectively.