FLDM-VTON: Faithful Latent Diffusion Model for Virtual Try-on

TL;DR

FLDM-VTON is a novel virtual try-on model that enhances faithfulness to clothing details by incorporating warped clothes, a clothes flattening network, and clothes-posterior sampling, leading to more realistic results.

Contribution

The paper introduces FLDM-VTON, a new latent diffusion approach that significantly improves clothing detail fidelity in virtual try-on images.

Findings

Outperforms state-of-the-art methods on VITON-HD and Dress Code datasets.

Generates photo-realistic try-on images with faithful clothing details.

Enhances model performance through clothes priors and specialized sampling.

Abstract

Despite their impressive generative performance, latent diffusion model-based virtual try-on (VTON) methods lack faithfulness to crucial details of the clothes, such as style, pattern, and text. To alleviate these issues caused by the diffusion stochastic nature and latent supervision, we propose a novel Faithful Latent Diffusion Model for VTON, termed FLDM-VTON. FLDM-VTON improves the conventional latent diffusion process in three major aspects. First, we propose incorporating warped clothes as both the starting point and local condition, supplying the model with faithful clothes priors. Second, we introduce a novel clothes flattening network to constrain generated try-on images, providing clothes-consistent faithful supervision. Third, we devise a clothes-posterior sampling for faithful inference, further enhancing the model performance over conventional clothes-agnostic Gaussian sampling. Extensive experimental results on the benchmark VITON-HD and Dress Code datasets demonstrate that our FLDM-VTON outperforms state-of-the-art baselines and is able to generate photo-realistic try-on images with faithful clothing details.