Fashionability-Enhancing Outfit Image Editing with Conditional Diffusion Models

TL;DR

This paper introduces a diffusion model that enhances the fashionability of outfit images while preserving key attributes, using expert-annotated scores and automatic optimization for improved style and appeal.

Contribution

The novel diffusion-based method improves fashionability in generated images with automatic guidance, maintaining original body characteristics without manual prompts.

Findings

Outperforms Fashion++ in fashionability scores

Generates more stylish and appealing outfit images

Uses expert-annotated data for guidance and evaluation

Abstract

Image generation in the fashion domain has predominantly focused on preserving body characteristics or following input prompts, but little attention has been paid to improving the inherent fashionability of the output images. This paper presents a novel diffusion model-based approach that generates fashion images with improved fashionability while maintaining control over key attributes. Key components of our method include: 1) fashionability enhancement, which ensures that the generated images are more fashionable than the input; 2) preservation of body characteristics, encouraging the generated images to maintain the original shape and proportions of the input; and 3) automatic fashion optimization, which does not rely on manual input or external prompts. We also employ two methods to collect training data for guidance while generating and evaluating the images. In particular, we rate outfit images using fashionability scores annotated by multiple fashion experts through OpenSkill-based and five critical aspect-based pairwise comparisons. These methods provide complementary perspectives for assessing and improving the fashionability of the generated images. The experimental results show that our approach outperforms the baseline Fashion++ in generating images with superior fashionability, demonstrating its effectiveness in producing more stylish and appealing fashion images.