WarpDiffusion: Efficient Diffusion Model for High-Fidelity Virtual Try-on

TL;DR

WarpDiffusion introduces an efficient diffusion-based approach for virtual try-on that enhances realism and detail preservation by combining warping and diffusion paradigms with novel attention and masking mechanisms.

Contribution

It proposes WarpDiffusion, a novel method that integrates local garment feature attention and auto-mask modules to improve high-fidelity virtual try-on while reducing resource consumption.

Findings

Outperforms state-of-the-art methods qualitatively and quantitatively

Achieves high-resolution, realistic virtual try-on results

Can be integrated into existing VITON frameworks

Abstract

Image-based Virtual Try-On (VITON) aims to transfer an in-shop garment image onto a target person. While existing methods focus on warping the garment to fit the body pose, they often overlook the synthesis quality around the garment-skin boundary and realistic effects like wrinkles and shadows on the warped garments. These limitations greatly reduce the realism of the generated results and hinder the practical application of VITON techniques. Leveraging the notable success of diffusion-based models in cross-modal image synthesis, some recent diffusion-based methods have ventured to tackle this issue. However, they tend to either consume a significant amount of training resources or struggle to achieve realistic try-on effects and retain garment details. For efficient and high-fidelity VITON, we propose WarpDiffusion, which bridges the warping-based and diffusion-based paradigms via a novel informative and local garment feature attention mechanism. Specifically, WarpDiffusion incorporates local texture attention to reduce resource consumption and uses a novel auto-mask module that effectively retains only the critical areas of the warped garment while disregarding unrealistic or erroneous portions. Notably, WarpDiffusion can be integrated as a plug-and-play component into existing VITON methodologies, elevating their synthesis quality. Extensive experiments on high-resolution VITON benchmarks and an in-the-wild test set demonstrate the superiority of WarpDiffusion, surpassing state-of-the-art methods both qualitatively and quantitatively.