M2E-Try On Net: Fashion from Model to Everyone

TL;DR

This paper introduces M2E-Try On Net, a novel virtual try-on system that transfers clothing from a model image to a person image without needing clean product images, using pose alignment, texture refinement, and fitting networks.

Contribution

The paper presents a self-supervised framework for realistic virtual try-on that handles pose, texture, and identity variations without requiring paired training data.

Findings

Achieves photo-realistic try-on results on Deep Fashion and MVC datasets.

Effectively handles diverse fashion items including upper and lower garments.

Outperforms existing methods in realism and detail preservation.

Abstract

Most existing virtual try-on applications require clean clothes images. Instead, we present a novel virtual Try-On network, M2E-Try On Net, which transfers the clothes from a model image to a person image without the need of any clean product images. To obtain a realistic image of person wearing the desired model clothes, we aim to solve the following challenges: 1) non-rigid nature of clothes - we need to align poses between the model and the user; 2) richness in textures of fashion items - preserving the fine details and characteristics of the clothes is critical for photo-realistic transfer; 3) variation of identity appearances - it is required to fit the desired model clothes to the person identity seamlessly. To tackle these challenges, we introduce three key components, including the pose alignment network (PAN), the texture refinement network (TRN) and the fitting network (FTN). Since it is unlikely to gather image pairs of input person image and desired output image (i.e. person wearing the desired clothes), our framework is trained in a self-supervised manner to gradually transfer the poses and textures of the model's clothes to the desired appearance. In the experiments, we verify on the Deep Fashion dataset and MVC dataset that our method can generate photo-realistic images for the person to try-on the model clothes. Furthermore, we explore the model capability for different fashion items, including both upper and lower garments.