STG-Avatar: Animatable Human Avatars via Spacetime Gaussian

TL;DR

STG-Avatar introduces a novel 3D Gaussian-based framework that combines rigid and non-rigid deformation techniques, enabling high-fidelity, real-time animatable human avatars from monocular videos, with improved detail and dynamic region handling.

Contribution

It proposes a hybrid deformation framework integrating Spacetime Gaussians with linear blend skinning for enhanced avatar reconstruction.

Findings

Outperforms state-of-the-art methods in reconstruction quality.

Achieves real-time rendering with high fidelity.

Effectively models dynamic clothing and limb movements.

Abstract

Realistic animatable human avatars from monocular videos are crucial for advancing human-robot interaction and enhancing immersive virtual experiences. While recent research on 3DGS-based human avatars has made progress, it still struggles with accurately representing detailed features of non-rigid objects (e.g., clothing deformations) and dynamic regions (e.g., rapidly moving limbs). To address these challenges, we present STG-Avatar, a 3DGS-based framework for high-fidelity animatable human avatar reconstruction. Specifically, our framework introduces a rigid-nonrigid coupled deformation framework that synergistically integrates Spacetime Gaussians (STG) with linear blend skinning (LBS). In this hybrid design, LBS enables real-time skeletal control by driving global pose transformations, while STG complements it through spacetime adaptive optimization of 3D Gaussians. Furthermore, we employ optical flow to identify high-dynamic regions and guide the adaptive densification of 3D Gaussians in these regions. Experimental results demonstrate that our method consistently outperforms state-of-the-art baselines in both reconstruction quality and operational efficiency, achieving superior quantitative metrics while retaining real-time rendering capabilities. Our code is available at https://github.com/jiangguangan/STG-Avatar