AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario

TL;DR

AnyFit introduces a scalable, robust virtual try-on system capable of generating high-fidelity, well-fitting garment images across diverse scenarios by leveraging a novel Hydra Block and residual synthesis techniques.

Contribution

The paper presents the Hydra Block for flexible attire combinations and a residual synthesis method to improve robustness and realism in virtual try-on applications.

Findings

Outperforms all baselines on high-resolution benchmarks

Generates photorealistic, well-fitting garments in diverse scenarios

Enhances robustness with residual synthesis and mask region strategies

Abstract

While image-based virtual try-on has made significant strides, emerging approaches still fall short of delivering high-fidelity and robust fitting images across various scenarios, as their models suffer from issues of ill-fitted garment styles and quality degrading during the training process, not to mention the lack of support for various combinations of attire. Therefore, we first propose a lightweight, scalable, operator known as Hydra Block for attire combinations. This is achieved through a parallel attention mechanism that facilitates the feature injection of multiple garments from conditionally encoded branches into the main network. Secondly, to significantly enhance the model's robustness and expressiveness in real-world scenarios, we evolve its potential across diverse settings by synthesizing the residuals of multiple models, as well as implementing a mask region boost strategy to overcome the instability caused by information leakage in existing models. Equipped with the above design, AnyFit surpasses all baselines on high-resolution benchmarks and real-world data by a large gap, excelling in producing well-fitting garments replete with photorealistic and rich details. Furthermore, AnyFit's impressive performance on high-fidelity virtual try-ons in any scenario from any image, paves a new path for future research within the fashion community.