ZFlow: Gated Appearance Flow-based Virtual Try-on with 3D Priors

TL;DR

ZFlow is an end-to-end virtual try-on framework that improves garment alignment and detail preservation by integrating hierarchical flow estimates and 3D structural priors, achieving state-of-the-art results.

Contribution

It introduces Gated Appearance Flow and dense 3D priors to enhance geometric and textural accuracy in virtual try-on, addressing limitations of previous two-stage methods.

Findings

Achieves superior image quality metrics (PSNR, SSIM, FID)

Outperforms existing methods qualitatively and quantitatively

Validated through extensive user studies and ablation analysis

Abstract

Image-based virtual try-on involves synthesizing perceptually convincing images of a model wearing a particular garment and has garnered significant research interest due to its immense practical applicability. Recent methods involve a two stage process: i) warping of the garment to align with the model ii) texture fusion of the warped garment and target model to generate the try-on output. Issues arise due to the non-rigid nature of garments and the lack of geometric information about the model or the garment. It often results in improper rendering of granular details. We propose ZFlow, an end-to-end framework, which seeks to alleviate these concerns regarding geometric and textural integrity (such as pose, depth-ordering, skin and neckline reproduction) through a combination of gated aggregation of hierarchical flow estimates termed Gated Appearance Flow, and dense structural priors at various stage of the network. ZFlow achieves state-of-the-art results as observed qualitatively, and on quantitative benchmarks of image quality (PSNR, SSIM, and FID). The paper presents extensive comparisons with other existing solutions including a detailed user study and ablation studies to gauge the effect of each of our contributions on multiple datasets.