LayerGS: Decomposition and Inpainting of Layered 3D Human Avatars via 2D Gaussian Splatting

TL;DR

LayerGS introduces a multi-layer 3D human avatar reconstruction framework that separates body and clothing, using 2D Gaussian splatting and diffusion-based inpainting to improve realism and flexibility in virtual try-on applications.

Contribution

The paper presents a novel multi-layer 3D human avatar reconstruction method that overcomes occlusion and clothing locking issues of prior approaches using 2D Gaussian encoding and diffusion inpainting.

Findings

Outperforms previous state-of-the-art in rendering quality and layer decomposition

Enables realistic virtual try-on from new viewpoints and poses

Achieves high-fidelity 3D human asset creation for immersive applications

Abstract

We propose a novel framework for decomposing arbitrarily posed humans into animatable multi-layered 3D human avatars, separating the body and garments. Conventional single-layer reconstruction methods lock clothing to one identity, while prior multi-layer approaches struggle with occluded regions. We overcome both limitations by encoding each layer as a set of 2D Gaussians for accurate geometry and photorealistic rendering, and inpainting hidden regions with a pretrained 2D diffusion model via score-distillation sampling (SDS). Our three-stage training strategy first reconstructs the coarse canonical garment via single-layer reconstruction, followed by multi-layer training to jointly recover the inner-layer body and outer-layer garment details. Experiments on two 3D human benchmark datasets (4D-Dress, Thuman2.0) show that our approach achieves better rendering quality and layer decomposition and recomposition than the previous state-of-the-art, enabling realistic virtual try-on under novel viewpoints and poses, and advancing practical creation of high-fidelity 3D human assets for immersive applications. Our code is available at https://github.com/RockyXu66/LayerGS