Guided and Variance-Corrected Fusion with One-shot Style Alignment for Large-Content Image Generation

TL;DR

This paper introduces novel fusion techniques, Guided Fusion and Variance-Corrected Fusion, along with a one-shot Style Alignment method, to improve large-content image generation using small diffusion models, reducing artifacts and enhancing style coherence.

Contribution

The paper presents new fusion and style alignment methods that significantly improve large image generation quality with small diffusion models, addressing artifacts and style inconsistency.

Findings

Fusion methods reduce seams and artifacts in large images.

Style alignment produces coherent styles without extra computation.

Experimental results show improved image quality across various tests.

Abstract

Producing large images using small diffusion models is gaining increasing popularity, as the cost of training large models could be prohibitive. A common approach involves jointly generating a series of overlapped image patches and obtaining large images by merging adjacent patches. However, results from existing methods often exhibit noticeable artifacts, e.g., seams and inconsistent objects and styles. To address the issues, we proposed Guided Fusion (GF), which mitigates the negative impact from distant image regions by applying a weighted average to the overlapping regions. Moreover, we proposed Variance-Corrected Fusion (VCF), which corrects data variance at post-averaging, generating more accurate fusion for the Denoising Diffusion Probabilistic Model. Furthermore, we proposed a one-shot Style Alignment (SA), which generates a coherent style for large images by adjusting the initial input noise without adding extra computational burden. Extensive experiments demonstrated that the proposed fusion methods improved the quality of the generated image significantly. The proposed method can be widely applied as a plug-and-play module to enhance other fusion-based methods for large image generation. Code: https://github.com/TitorX/GVCFDiffusion