Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images

TL;DR

This paper introduces Any-Size-Diffusion, a two-stage method that efficiently generates high-quality images of any size from text prompts, reducing memory usage and inference time compared to existing approaches.

Contribution

The paper proposes a novel two-stage pipeline combining ARAD and FSTD to enable flexible, high-resolution image synthesis with minimal computational resources.

Findings

ASD produces well-structured images of arbitrary sizes.

Inference time is reduced by 2x compared to traditional tiled methods.

The approach demonstrates effectiveness on LAION-COCO and MM-CelebA-HQ benchmarks.

Abstract

Stable diffusion, a generative model used in text-to-image synthesis, frequently encounters resolution-induced composition problems when generating images of varying sizes. This issue primarily stems from the model being trained on pairs of single-scale images and their corresponding text descriptions. Moreover, direct training on images of unlimited sizes is unfeasible, as it would require an immense number of text-image pairs and entail substantial computational expenses. To overcome these challenges, we propose a two-stage pipeline named Any-Size-Diffusion (ASD), designed to efficiently generate well-composed images of any size, while minimizing the need for high-memory GPU resources. Specifically, the initial stage, dubbed Any Ratio Adaptability Diffusion (ARAD), leverages a selected set of images with a restricted range of ratios to optimize the text-conditional diffusion model, thereby improving its ability to adjust composition to accommodate diverse image sizes. To support the creation of images at any desired size, we further introduce a technique called Fast Seamless Tiled Diffusion (FSTD) at the subsequent stage. This method allows for the rapid enlargement of the ASD output to any high-resolution size, avoiding seaming artifacts or memory overloads. Experimental results on the LAION-COCO and MM-CelebA-HQ benchmarks demonstrate that ASD can produce well-structured images of arbitrary sizes, cutting down the inference time by 2x compared to the traditional tiled algorithm.