Real-Time Per-Garment Virtual Try-On with Temporal Consistency for Loose-Fitting Garments

TL;DR

This paper presents a real-time virtual try-on system for loose-fitting garments that enhances semantic map estimation and incorporates temporal coherence to improve image quality and reduce jittering artifacts.

Contribution

It introduces a two-stage semantic map estimation method and a recurrent synthesis framework for better temporal consistency in virtual try-on.

Findings

Outperforms existing methods in image quality

Achieves superior temporal coherence

Maintains real-time performance

Abstract

Per-garment virtual try-on methods collect garment-specific datasets and train networks tailored to each garment to achieve superior results. However, these approaches often struggle with loose-fitting garments due to two key limitations: (1) They rely on human body semantic maps to align garments with the body, but these maps become unreliable when body contours are obscured by loose-fitting garments, resulting in degraded outcomes; (2) They train garment synthesis networks on a per-frame basis without utilizing temporal information, leading to noticeable jittering artifacts. To address the first limitation, we propose a two-stage approach for robust semantic map estimation. First, we extract a garment-invariant representation from the raw input image. This representation is then passed through an auxiliary network to estimate the semantic map. This enhances the robustness of semantic map estimation under loose-fitting garments during garment-specific dataset generation. To address the second limitation, we introduce a recurrent garment synthesis framework that incorporates temporal dependencies to improve frame-to-frame coherence while maintaining real-time performance. We conducted qualitative and quantitative evaluations to demonstrate that our method outperforms existing approaches in both image quality and temporal coherence. Ablation studies further validate the effectiveness of the garment-invariant representation and the recurrent synthesis framework.