Compatible and Diverse Fashion Image Inpainting

TL;DR

This paper introduces FiNet, a two-stage fashion image inpainting framework that generates compatible, diverse, and photorealistic fashion images by disentangling shape and appearance and learning a shared compatibility space.

Contribution

The paper proposes a novel two-stage inpainting framework with shared latent space learning for compatibility and diversity in fashion image synthesis.

Findings

Outperforms state-of-the-art methods in fashion synthesis tasks.

Produces highly compatible and diverse fashion inpainting results.

Extends effectively to clothing reconstruction and fashion transfer.

Abstract

Visual compatibility is critical for fashion analysis, yet is missing in existing fashion image synthesis systems. In this paper, we propose to explicitly model visual compatibility through fashion image inpainting. To this end, we present Fashion Inpainting Networks (FiNet), a two-stage image-to-image generation framework that is able to perform compatible and diverse inpainting. Disentangling the generation of shape and appearance to ensure photorealistic results, our framework consists of a shape generation network and an appearance generation network. More importantly, for each generation network, we introduce two encoders interacting with one another to learn latent code in a shared compatibility space. The latent representations are jointly optimized with the corresponding generation network to condition the synthesis process, encouraging a diverse set of generated results that are visually compatible with existing fashion garments. In addition, our framework is readily extended to clothing reconstruction and fashion transfer, with impressive results. Extensive experiments with comparisons with state-of-the-art approaches on fashion synthesis task quantitatively and qualitatively demonstrate the effectiveness of our method.