Design Booster: A Text-Guided Diffusion Model for Image Translation with Spatial Layout Preservation

TL;DR

Design Booster introduces a flexible, layout-aware diffusion model for image translation that preserves spatial structure, allows multi-condition control, and outperforms existing methods in quality and speed.

Contribution

It proposes a novel training framework that co-encodes images and text for flexible, layout-preserving image translation with efficient inference.

Findings

Outperforms state-of-the-art in style and semantic translation

Achieves faster inference times

Maintains spatial layout effectively

Abstract

Diffusion models are able to generate photorealistic images in arbitrary scenes. However, when applying diffusion models to image translation, there exists a trade-off between maintaining spatial structure and high-quality content. Besides, existing methods are mainly based on test-time optimization or fine-tuning model for each input image, which are extremely time-consuming for practical applications. To address these issues, we propose a new approach for flexible image translation by learning a layout-aware image condition together with a text condition. Specifically, our method co-encodes images and text into a new domain during the training phase. In the inference stage, we can choose images/text or both as the conditions for each time step, which gives users more flexible control over layout and content. Experimental comparisons of our method with state-of-the-art methods demonstrate our model performs best in both style image translation and semantic image translation and took the shortest time.