TwinDiffusion: Enhancing Coherence and Efficiency in Panoramic Image Generation with Diffusion Models

TL;DR

TwinDiffusion is a novel diffusion-based framework that significantly improves the coherence, quality, and efficiency of high-resolution panoramic image generation through innovative fusion and sampling techniques.

Contribution

The paper introduces TwinDiffusion, a new diffusion model framework with crop fusion and cross sampling strategies, enhancing panoramic image coherence and efficiency without additional training.

Findings

Outperforms prior methods in coherence and fidelity

Achieves higher efficiency in panoramic image generation

Produces seamless, high-quality panoramic images

Abstract

Diffusion models have emerged as effective tools for generating diverse and high-quality content. However, their capability in high-resolution image generation, particularly for panoramic images, still faces challenges such as visible seams and incoherent transitions. In this paper, we propose TwinDiffusion, an optimized framework designed to address these challenges through two key innovations: the Crop Fusion for quality enhancement and the Cross Sampling for efficiency optimization. We introduce a training-free optimizing stage to refine the similarity of adjacent image areas, as well as an interleaving sampling strategy to yield dynamic patches during the cropping process. A comprehensive evaluation is conducted to compare TwinDiffusion with the prior works, considering factors including coherence, fidelity, compatibility, and efficiency. The results demonstrate the superior performance of our approach in generating seamless and coherent panoramas, setting a new standard in quality and efficiency for panoramic image generation.