Learning Implicit Features with Flow Infused Attention for Realistic Virtual Try-On

TL;DR

FIA-VTON introduces a flow-infused attention mechanism that implicitly guides garment fitting in virtual try-on, improving robustness and detail preservation over explicit warping methods.

Contribution

The paper proposes a novel implicit warp feature using flow-infused attention, reducing reliance on accurate warping and enhancing virtual try-on quality.

Findings

Outperforms state-of-the-art methods on VTON-HD and DressCode datasets.

Demonstrates robustness to warping estimation errors.

Effectively preserves garment details and fit.

Abstract

Image-based virtual try-on is challenging since the generated image should fit the garment to model images in various poses and keep the characteristics and details of the garment simultaneously. A popular research stream warps the garment image firstly to reduce the burden of the generation stage, which relies highly on the performance of the warping module. Other methods without explicit warping often lack sufficient guidance to fit the garment to the model images. In this paper, we propose FIA-VTON, which leverages the implicit warp feature by adopting a Flow Infused Attention module on virtual try-on. The dense warp flow map is projected as indirect guidance attention to enhance the feature map warping in the generation process implicitly, which is less sensitive to the warping estimation accuracy than an explicit warp of the garment image. To further enhance implicit warp guidance, we incorporate high-level spatial attention to complement the dense warp. Experimental results on the VTON-HD and DressCode dataset significantly outperform state-of-the-art methods, demonstrating that FIA-VTON is effective and robust for virtual try-on.