Generative Panoramic Image Stitching

TL;DR

This paper presents a novel generative approach for panoramic image stitching that synthesizes seamless, coherent panoramas from multiple reference images, overcoming limitations of traditional methods and recent generative models.

Contribution

It introduces a fine-tuned diffusion-based inpainting model for panoramic stitching that preserves scene content and layout across multiple reference images.

Findings

Outperforms baselines in image quality and scene consistency

Produces seamless panoramas from a single reference image

Effectively handles parallax and lighting variations

Abstract

We introduce the task of generative panoramic image stitching, which aims to synthesize seamless panoramas that are faithful to the content of multiple reference images containing parallax effects and strong variations in lighting, camera capture settings, or style. In this challenging setting, traditional image stitching pipelines fail, producing outputs with ghosting and other artifacts. While recent generative models are capable of outpainting content consistent with multiple reference images, they fail when tasked with synthesizing large, coherent regions of a panorama. To address these limitations, we propose a method that fine-tunes a diffusion-based inpainting model to preserve a scene's content and layout based on multiple reference images. Once fine-tuned, the model outpaints a full panorama from a single reference image, producing a seamless and visually coherent result that faithfully integrates content from all reference images. Our approach significantly outperforms baselines for this task in terms of image quality and the consistency of image structure and scene layout when evaluated on captured datasets.