GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details

TL;DR

GarmentDreamer is a novel 3D garment synthesis method that uses 3D Gaussian Splatting guidance to produce diverse, consistent, and detailed wearable garments from text prompts, overcoming limitations of previous approaches.

Contribution

The paper introduces GarmentDreamer, a new approach leveraging 3D Gaussian Splatting guidance and implicit Neural Texture Fields for high-quality, diverse 3D garment generation from text prompts.

Findings

Outperforms state-of-the-art methods in qualitative and quantitative evaluations.

Produces consistent multi-view garment meshes with rich geometric and texture details.

Enables efficient and flexible 3D garment synthesis from text descriptions.

Abstract

Traditional 3D garment creation is labor-intensive, involving sketching, modeling, UV mapping, and texturing, which are time-consuming and costly. Recent advances in diffusion-based generative models have enabled new possibilities for 3D garment generation from text prompts, images, and videos. However, existing methods either suffer from inconsistencies among multi-view images or require additional processes to separate cloth from the underlying human model. In this paper, we propose GarmentDreamer, a novel method that leverages 3D Gaussian Splatting (GS) as guidance to generate wearable, simulation-ready 3D garment meshes from text prompts. In contrast to using multi-view images directly predicted by generative models as guidance, our 3DGS guidance ensures consistent optimization in both garment deformation and texture synthesis. Our method introduces a novel garment augmentation module, guided by normal and RGBA information, and employs implicit Neural Texture Fields (NeTF) combined with Score Distillation Sampling (SDS) to generate diverse geometric and texture details. We validate the effectiveness of our approach through comprehensive qualitative and quantitative experiments, showcasing the superior performance of GarmentDreamer over state-of-the-art alternatives. Our project page is available at: https://xuan-li.github.io/GarmentDreamerDemo/.