Training-free Clothing Region of Interest Self-correction for Virtual Try-On

TL;DR

This paper introduces a training-free method for virtual try-on that enhances clothing detail accuracy using an energy-based attention constraint and proposes a new comprehensive evaluation metric, VTID, outperforming previous methods on multiple datasets.

Contribution

It presents a novel training-free approach with an energy function for focused attention in virtual try-on, along with a new metric VTID for better evaluation.

Findings

Outperforms SOTA in LPIPS, FID, KID, and VTID metrics.

Improves downstream clothing re-identification performance.

Demonstrates effectiveness on VITON-HD and DressCode datasets.

Abstract

VTON (Virtual Try-ON) aims at synthesizing the target clothing on a certain person, preserving the details of the target clothing while keeping the rest of the person unchanged. Existing methods suffer from the discrepancies between the generated clothing results and the target ones, in terms of the patterns, textures and boundaries. Therefore, we propose to use an energy function to impose constraints on the attention map extracted through the generation process. Thus, at each generation step, the attention can be more focused on the clothing region of interest, thereby influencing the generation results to be more consistent with the target clothing details. Furthermore, to address the limitation that existing evaluation metrics concentrate solely on image realism and overlook the alignment with target elements, we design a new metric, Virtual Try-on Inception Distance (VTID), to bridge this gap and ensure a more comprehensive assessment. On the VITON-HD and DressCode datasets, our approach has outperformed the previous state-of-the-art (SOTA) methods by 1.4%, 2.3%, 12.3%, and 5.8% in the traditional metrics of LPIPS, FID, KID, and the new VTID metrics, respectively. Additionally, by applying the generated data to downstream Clothing-Change Re-identification (CC-Reid) methods, we have achieved performance improvements of 2.5%, 1.1%, and 1.6% on the LTCC, PRCC, VC-Clothes datasets in the metrics of Rank-1. The code of our method is public at https://github.com/MrWhiteSmall/CSC-VTON.git.