Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video

TL;DR

This paper presents Gaussian Garments, a method for reconstructing realistic, simulation-ready clothing assets from multi-view videos using a combined 3D mesh and Gaussian texture representation, enabling detailed appearance and behavior modeling.

Contribution

It introduces a novel Gaussian texture representation for garments and a fine-tuning approach for GNNs to replicate garment behavior from multi-view video data.

Findings

Accurate registration of garment geometries to multi-view videos.

Effective disentanglement of albedo and lighting effects.

Automatic assembly and animation of multi-garment outfits.

Abstract

We introduce Gaussian Garments, a novel approach for reconstructing realistic simulation-ready garment assets from multi-view videos. Our method represents garments with a combination of a 3D mesh and a Gaussian texture that encodes both the color and high-frequency surface details. This representation enables accurate registration of garment geometries to multi-view videos and helps disentangle albedo textures from lighting effects. Furthermore, we demonstrate how a pre-trained graph neural network (GNN) can be fine-tuned to replicate the real behavior of each garment. The reconstructed Gaussian Garments can be automatically combined into multi-garment outfits and animated with the fine-tuned GNN.