Gaussian Wardrobe: Compositional 3D Gaussian Avatars for Free-Form Virtual Try-On

TL;DR

Gaussian Wardrobe introduces a compositional 3D avatar framework that disentangles body and clothing layers from multi-view videos, enabling realistic free-form garment modeling and versatile virtual try-on applications.

Contribution

It proposes a novel shape-agnostic, compositional 3D Gaussian representation for avatars, allowing independent garment modeling and reuse across different subjects.

Findings

Achieves state-of-the-art results in pose synthesis benchmarks.

Models photorealistic avatars with high-fidelity dynamics.

Enables practical virtual try-on with clothing transfer to new subjects.

Abstract

We introduce Gaussian Wardrobe, a novel framework to digitalize compositional 3D neural avatars from multi-view videos. Existing methods for 3D neural avatars typically treat the human body and clothing as an inseparable entity. However, this paradigm fails to capture the dynamics of complex free-form garments and limits the reuse of clothing across different individuals. To overcome these problems, we develop a novel, compositional 3D Gaussian representation to build avatars from multiple layers of free-form garments. The core of our method is decomposing neural avatars into bodies and layers of shape-agnostic neural garments. To achieve this, our framework learns to disentangle each garment layer from multi-view videos and canonicalizes it into a shape-independent space. In experiments, our method models photorealistic avatars with high-fidelity dynamics, achieving new state-of-the-art performance on novel pose synthesis benchmarks. In addition, we demonstrate that the learned compositional garments contribute to a versatile digital wardrobe, enabling a practical virtual try-on application where clothing can be freely transferred to new subjects. Project page: https://ait.ethz.ch/gaussianwardrobe