Multi-Garment Customized Model Generation

TL;DR

This paper presents a novel framework using Latent Diffusion Models for generating images of models wearing multiple garments based on text prompts, addressing challenges in preserving garment details and natural appearance.

Contribution

The paper introduces a multi-garment generation framework with a trainable garment encoder and decoupled feature fusion, enabling flexible and high-quality multi-clothing image synthesis.

Findings

Outperforms existing methods in multi-garment image synthesis

Supports diverse clothing combinations with high fidelity

Enhances controllability through plug-and-play modules

Abstract

This paper introduces Multi-Garment Customized Model Generation, a unified framework based on Latent Diffusion Models (LDMs) aimed at addressing the unexplored task of synthesizing images with free combinations of multiple pieces of clothing. The method focuses on generating customized models wearing various targeted outfits according to different text prompts. The primary challenge lies in maintaining the natural appearance of the dressed model while preserving the complex textures of each piece of clothing, ensuring that the information from different garments does not interfere with each other. To tackle these challenges, we first developed a garment encoder, which is a trainable UNet copy with shared weights, capable of extracting detailed features of garments in parallel. Secondly, our framework supports the conditional generation of multiple garments through decoupled multi-garment feature fusion, allowing multiple clothing features to be injected into the backbone network, significantly alleviating conflicts between garment information. Additionally, the proposed garment encoder is a plug-and-play module that can be combined with other extension modules such as IP-Adapter and ControlNet, enhancing the diversity and controllability of the generated models. Extensive experiments demonstrate the superiority of our approach over existing alternatives, opening up new avenues for the task of generating images with multiple-piece clothing combinations