Low-Barrier Dataset Collection with Real Human Body for Interactive Per-Garment Virtual Try-On

TL;DR

This paper introduces a low-barrier, real-human-based dataset collection method and a hybrid representation for virtual try-on, improving alignment, realism, and user interaction without expensive robotic mannequins.

Contribution

It presents a novel dataset collection approach using real human bodies and a hybrid representation with DensePose, enhancing garment alignment and interaction in virtual try-on.

Findings

Outperforms state-of-the-art methods in image quality and temporal consistency.

User study shows high helpfulness for garment purchasing decisions.

Achieves accurate garment alignment without specialized wearable devices.

Abstract

Existing image-based virtual try-on methods are often limited to the front view and lack real-time performance. While per-garment virtual try-on methods have tackled these issues by capturing per-garment datasets and training per-garment neural networks, they still encounter practical limitations: (1) the robotic mannequin used to capture per-garment datasets is prohibitively expensive for widespread adoption and fails to accurately replicate natural human body deformation; (2) the synthesized garments often misalign with the human body. To address these challenges, we propose a low-barrier approach for collecting per-garment datasets using real human bodies, eliminating the necessity for a customized robotic mannequin. We also introduce a hybrid person representation that enhances the existing intermediate representation with a simplified DensePose map. This ensures accurate alignment of synthesized garment images with the human body and enables human-garment interaction without the need for customized wearable devices. We performed qualitative and quantitative evaluations against other state-of-the-art image-based virtual try-on methods and conducted ablation studies to demonstrate the superiority of our method regarding image quality and temporal consistency. Finally, our user study results indicated that most participants found our virtual try-on system helpful for making garment purchasing decisions.