TexControl: Sketch-Based Two-Stage Fashion Image Generation Using Diffusion Model

TL;DR

TexControl is a two-stage diffusion-based framework that converts sketches into detailed fashion images, effectively capturing textures and outlines for high-quality design generation.

Contribution

It introduces a novel two-stage pipeline combining ControlNet and image-to-image methods for improved sketch-based fashion image synthesis.

Findings

Generates high-quality fashion images with detailed textures.

Maintains stable outlines from sketches during generation.

Outperforms existing methods in texture detail and image quality.

Abstract

Deep learning-based sketch-to-clothing image generation provides the initial designs and inspiration in the fashion design processes. However, clothing generation from freehand drawing is challenging due to the sparse and ambiguous information from the drawn sketches. The current generation models may have difficulty generating detailed texture information. In this work, we propose TexControl, a sketch-based fashion generation framework that uses a two-stage pipeline to generate the fashion image corresponding to the sketch input. First, we adopt ControlNet to generate the fashion image from sketch and keep the image outline stable. Then, we use an image-to-image method to optimize the detailed textures of the generated images and obtain the final results. The evaluation results show that TexControl can generate fashion images with high-quality texture as fine-grained image generation.