Language Guided Fashion Image Manipulation with Feature-wise Transformations

TL;DR

This paper introduces FiLMedGAN, a novel method for editing fashion images based on natural language descriptions, achieving realistic and well-localized outfit modifications without extra spatial data.

Contribution

The work presents FiLMedGAN, which uses feature-wise linear modulation to relate language and visual features for precise, realistic fashion image editing, outperforming baseline methods.

Findings

Produces more plausible outfit edits than baseline

Demonstrates better localization of modifications

Achieves realistic image generation with natural language guidance

Abstract

Developing techniques for editing an outfit image through natural sentences and accordingly generating new outfits has promising applications for art, fashion and design. However, it is considered as a certainly challenging task since image manipulation should be carried out only on the relevant parts of the image while keeping the remaining sections untouched. Moreover, this manipulation process should generate an image that is as realistic as possible. In this work, we propose FiLMedGAN, which leverages feature-wise linear modulation (FiLM) to relate and transform visual features with natural language representations without using extra spatial information. Our experiments demonstrate that this approach, when combined with skip connections and total variation regularization, produces more plausible results than the baseline work, and has a better localization capability when generating new outfits consistent with the target description.