FitControler: Toward Fit-Aware Virtual Try-On

TL;DR

This paper introduces FitControler, a plug-in for virtual try-on systems that enables customizable garment fit control by generating fit-aware layouts and rendering, supported by a new dataset and fit consistency metrics.

Contribution

The paper presents a novel fit-aware VTON method with a learnable plug-in, a new dataset, and fit consistency metrics, advancing the realism and customization in virtual try-on applications.

Findings

FitControler seamlessly integrates with existing VTON models.

It achieves accurate fit control across various garment styles.

The approach improves the realism and customization of virtual try-on results.

Abstract

Realistic virtual try-on (VTON) concerns not only faithful rendering of garment details but also coordination of the style. Prior art typically pursues the former, but neglects a key factor that shapes the holistic style -- garment fit. Garment fit delineates how a garment aligns with the body of a wearer and is a fundamental element in fashion design. In this work, we introduce fit-aware VTON and present FitControler, a learnable plug-in that can seamlessly integrate into modern VTON models to enable customized fit control. To achieve this, we highlight two challenges: i) how to delineate layouts of different fits and ii) how to render the garment that matches the layout. FitControler first features a fit-aware layout generator to redraw the body-garment layout conditioned on a set of delicately processed garment-agnostic representations, and a multi-scale fit injector is then used to deliver layout cues to enable layout-driven VTON. In particular, we build a fit-aware VTON dataset termed Fit4Men, including 13,000 body-garment pairs of different fits, covering both tops and bottoms, and featuring varying camera distances and body poses. Two fit consistency metrics are also introduced to assess the fitness of generations. Extensive experiments show that FitControler can work with various VTON models and achieve accurate fit control. Code and data will be released.