PASTA-GAN++: A Versatile Framework for High-Resolution Unpaired Virtual Try-on

TL;DR

PASTA-GAN++ is a versatile high-resolution unpaired virtual try-on framework that supports unsupervised training, arbitrary garment categories, and controllable editing through innovative patch-based and spatially-adaptive modules.

Contribution

It introduces a novel patch-routed disentanglement and spatially-adaptive residual modules for high-resolution unpaired virtual try-on, enabling style preservation, category flexibility, and editing control.

Findings

Outperforms existing state-of-the-art methods on UPT dataset.

Supports arbitrary garment categories and local garment editing.

Produces realistic texture details in try-on results.

Abstract

Image-based virtual try-on is one of the most promising applications of human-centric image generation due to its tremendous real-world potential. In this work, we take a step forwards to explore versatile virtual try-on solutions, which we argue should possess three main properties, namely, they should support unsupervised training, arbitrary garment categories, and controllable garment editing. To this end, we propose a characteristic-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN++ (PASTA-GAN++), to achieve a versatile system for high-resolution unpaired virtual try-on. Specifically, our PASTA-GAN++ consists of an innovative patch-routed disentanglement module to decouple the intact garment into normalized patches, which is capable of retaining garment style information while eliminating the garment spatial information, thus alleviating the overfitting issue during unsupervised training. Furthermore, PASTA-GAN++ introduces a patch-based garment representation and a patch-guided parsing synthesis block, allowing it to handle arbitrary garment categories and support local garment editing. Finally, to obtain try-on results with realistic texture details, PASTA-GAN++ incorporates a novel spatially-adaptive residual module to inject the coarse warped garment feature into the generator. Extensive experiments on our newly collected UnPaired virtual Try-on (UPT) dataset demonstrate the superiority of PASTA-GAN++ over existing SOTAs and its ability for controllable garment editing.