Human Gaussian Splatting: Real-time Rendering of Animatable Avatars

TL;DR

This paper introduces Human Gaussian Splatting, a real-time, animatable human avatar rendering method using 3D Gaussian primitives that outperforms previous neural models in quality and speed.

Contribution

It presents a novel end-to-end trainable Gaussian Splatting model for animatable human avatars, combining efficiency with high visual fidelity.

Findings

Achieves 1.5 dB PSNR improvement over state-of-the-art methods.

Supports real-time rendering at 80 fps for 512x512 resolution.

Effective in novel pose synthesis of clothed human bodies.

Abstract

This work addresses the problem of real-time rendering of photorealistic human body avatars learned from multi-view videos. While the classical approaches to model and render virtual humans generally use a textured mesh, recent research has developed neural body representations that achieve impressive visual quality. However, these models are difficult to render in real-time and their quality degrades when the character is animated with body poses different than the training observations. We propose an animatable human model based on 3D Gaussian Splatting, that has recently emerged as a very efficient alternative to neural radiance fields. The body is represented by a set of gaussian primitives in a canonical space which is deformed with a coarse to fine approach that combines forward skinning and local non-rigid refinement. We describe how to learn our Human Gaussian Splatting (HuGS) model in an end-to-end fashion from multi-view observations, and evaluate it against the state-of-the-art approaches for novel pose synthesis of clothed body. Our method achieves 1.5 dB PSNR improvement over the state-of-the-art on THuman4 dataset while being able to render in real-time (80 fps for 512x512 resolution).