Diffusion360: Seamless 360 Degree Panoramic Image Generation based on Diffusion Models

TL;DR

This paper introduces Diffusion360, a novel diffusion-based method for generating seamless 360-degree panoramic images, addressing the challenge of maintaining geometry continuity through a circular blending strategy during diffusion and decoding stages.

Contribution

It proposes a circular blending strategy for diffusion models to generate seamless 360-degree panoramas, along with two models for text-to-360 and single-image-to-360 panorama generation.

Findings

Effective seamless 360 panorama generation demonstrated

Open-source code available for reproducibility

Addresses the continuity challenge in diffusion-based panoramic synthesis

Abstract

This is a technical report on the 360-degree panoramic image generation task based on diffusion models. Unlike ordinary 2D images, 360-degree panoramic images capture the entire $360^\circ\times 180^\circ$ field of view. So the rightmost and the leftmost sides of the 360 panoramic image should be continued, which is the main challenge in this field. However, the current diffusion pipeline is not appropriate for generating such a seamless 360-degree panoramic image. To this end, we propose a circular blending strategy on both the denoising and VAE decoding stages to maintain the geometry continuity. Based on this, we present two models for \textbf{Text-to-360-panoramas} and \textbf{Single-Image-to-360-panoramas} tasks. The code has been released as an open-source project at \href{https://github.com/ArcherFMY/SD-T2I-360PanoImage}{https://github.com/ArcherFMY/SD-T2I-360PanoImage} and \href{https://www.modelscope.cn/models/damo/cv_diffusion_text-to-360panorama-image_generation/summary}{ModelScope}