VTONQA: A Multi-Dimensional Quality Assessment Dataset for Virtual Try-on

TL;DR

VTONQA is a comprehensive dataset for evaluating the quality of virtual try-on images across multiple dimensions, aiding the development of better quality assessment methods and VTON models.

Contribution

This paper introduces the first multi-dimensional quality assessment dataset specifically for VTON, with extensive annotations and benchmarks for model evaluation.

Findings

Existing IQA metrics have limitations in VTON quality assessment.

VTONQA provides detailed annotations across three key quality dimensions.

Benchmark results highlight the need for specialized VTON quality assessment methods.

Abstract

With the rapid development of e-commerce and digital fashion, image-based virtual try-on (VTON) has attracted increasing attention. However, existing VTON models often suffer from artifacts such as garment distortion and body inconsistency, highlighting the need for reliable quality evaluation of VTON-generated images. To this end, we construct VTONQA, the first multi-dimensional quality assessment dataset specifically designed for VTON, which contains 8,132 images generated by 11 representative VTON models, along with 24,396 mean opinion scores (MOSs) across three evaluation dimensions (i.e., clothing fit, body compatibility, and overall quality). Based on VTONQA, we benchmark both VTON models and a diverse set of image quality assessment (IQA) metrics, revealing the limitations of existing methods and highlighting the value of the proposed dataset. We believe that the VTONQA dataset and corresponding benchmarks will provide a solid foundation for perceptually aligned evaluation, benefiting both the development of quality assessment methods and the advancement of VTON models.