OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on

TL;DR

OOTDiffusion introduces a new latent diffusion-based network architecture for realistic and controllable virtual try-on, eliminating warping steps and enabling adjustable garment features for high-quality image synthesis.

Contribution

The paper proposes OOTDiffusion, a novel outfitting UNet with self-attention and dropout mechanisms, enhancing controllability and realism in virtual try-on without redundant warping.

Findings

Outperforms existing VTON methods in realism and controllability

Generates high-quality try-on results for diverse human and garment images

Demonstrates effectiveness on VITON-HD and Dress Code datasets

Abstract

We present OOTDiffusion, a novel network architecture for realistic and controllable image-based virtual try-on (VTON). We leverage the power of pretrained latent diffusion models, designing an outfitting UNet to learn the garment detail features. Without a redundant warping process, the garment features are precisely aligned with the target human body via the proposed outfitting fusion in the self-attention layers of the denoising UNet. In order to further enhance the controllability, we introduce outfitting dropout to the training process, which enables us to adjust the strength of the garment features through classifier-free guidance. Our comprehensive experiments on the VITON-HD and Dress Code datasets demonstrate that OOTDiffusion efficiently generates high-quality try-on results for arbitrary human and garment images, which outperforms other VTON methods in both realism and controllability, indicating an impressive breakthrough in virtual try-on. Our source code is available at https://github.com/levihsu/OOTDiffusion.