From Design Draft to Real Attire: Unaligned Fashion Image Translation

TL;DR

This paper introduces D2RNet, a novel unaligned translation method that converts fashion design drafts into realistic clothing images by addressing large modality misalignments through adaptive sampling and dual translation streams.

Contribution

It proposes a new unaligned translation framework with a sampling network and dual streams for shape and texture, improving realism and consistency in fashion image synthesis.

Findings

Outperforms state-of-the-art methods in unaligned fashion translation

Generates realistic garments with shape and texture consistency

Effective application to reverse translation (real to draft)

Abstract

Fashion manipulation has attracted growing interest due to its great application value, which inspires many researches towards fashion images. However, little attention has been paid to fashion design draft. In this paper, we study a new unaligned translation problem between design drafts and real fashion items, whose main challenge lies in the huge misalignment between the two modalities. We first collect paired design drafts and real fashion item images without pixel-wise alignment. To solve the misalignment problem, our main idea is to train a sampling network to adaptively adjust the input to an intermediate state with structure alignment to the output. Moreover, built upon the sampling network, we present design draft to real fashion item translation network (D2RNet), where two separate translation streams that focus on texture and shape, respectively, are combined tactfully to get both benefits. D2RNet is able to generate realistic garments with both texture and shape consistency to their design drafts. We show that this idea can be effectively applied to the reverse translation problem and present R2DNet accordingly. Extensive experiments on unaligned fashion design translation demonstrate the superiority of our method over state-of-the-art methods. Our project website is available at: https://victoriahy.github.io/MM2020/ .