DAGSM: Disentangled Avatar Generation with GS-enhanced Mesh

TL;DR

DAGSM introduces a novel text-driven avatar generation pipeline that creates disentangled human bodies and garments using GS-enhanced meshes, enabling better clothing customization and realistic animations.

Contribution

The paper presents a new method for generating disentangled avatars with GS-enhanced meshes, improving clothing separation, texture quality, and animation realism compared to prior approaches.

Findings

Supports clothing replacement and realistic animation

Outperforms baselines in visual quality

Generates high-quality disentangled avatars

Abstract

Text-driven avatar generation has gained significant attention owing to its convenience. However, existing methods typically model the human body with all garments as a single 3D model, limiting its usability, such as clothing replacement, and reducing user control over the generation process. To overcome the limitations above, we propose DAGSM, a novel pipeline that generates disentangled human bodies and garments from the given text prompts. Specifically, we model each part (e.g., body, upper/lower clothes) of the clothed human as one GS-enhanced mesh (GSM), which is a traditional mesh attached with 2D Gaussians to better handle complicated textures (e.g., woolen, translucent clothes) and produce realistic cloth animations. During the generation, we first create the unclothed body, followed by a sequence of individual cloth generation based on the body, where we introduce a semantic-based algorithm to achieve better human-cloth and garment-garment separation. To improve texture quality, we propose a view-consistent texture refinement module, including a cross-view attention mechanism for texture style consistency and an incident-angle-weighted denoising (IAW-DE) strategy to update the appearance. Extensive experiments have demonstrated that DAGSM generates high-quality disentangled avatars, supports clothing replacement and realistic animation, and outperforms the baselines in visual quality.