SpotDiffusion: A Fast Approach For Seamless Panorama Generation Over Time

TL;DR

SpotDiffusion introduces a novel, efficient method for seamless panorama generation using diffusion models, significantly reducing computational costs and inference time while maintaining high image quality.

Contribution

The paper proposes a new approach that avoids multiple overlapping predictions, enabling faster and more efficient panorama generation with diffusion models.

Findings

Achieves comparable or better image quality than existing methods.

Reduces computational complexity and inference time.

Produces seamless high-resolution panoramas effectively.

Abstract

Generating high-resolution images with generative models has recently been made widely accessible by leveraging diffusion models pre-trained on large-scale datasets. Various techniques, such as MultiDiffusion and SyncDiffusion, have further pushed image generation beyond training resolutions, i.e., from square images to panorama, by merging multiple overlapping diffusion paths or employing gradient descent to maintain perceptual coherence. However, these methods suffer from significant computational inefficiencies due to generating and averaging numerous predictions, which is required in practice to produce high-quality and seamless images. This work addresses this limitation and presents a novel approach that eliminates the need to generate and average numerous overlapping denoising predictions. Our method shifts non-overlapping denoising windows over time, ensuring that seams in one timestep are corrected in the next. This results in coherent, high-resolution images with fewer overall steps. We demonstrate the effectiveness of our approach through qualitative and quantitative evaluations, comparing it with MultiDiffusion, SyncDiffusion, and StitchDiffusion. Our method offers several key benefits, including improved computational efficiency and faster inference times while producing comparable or better image quality. Link to code https://github.com/stanifrolov/spotdiffusion