Conditional Panoramic Image Generation via Masked Autoregressive Modeling

TL;DR

This paper introduces PAR, a unified autoregressive model for panoramic image generation that overcomes diffusion model limitations and seamlessly integrates text and image conditioning, improving coherence and versatility.

Contribution

The paper presents a novel masked autoregressive framework for panoramic images that unifies text and image conditioned generation, addressing diffusion model limitations and enhancing spatial coherence.

Findings

Competitive results in text-to-image generation and panorama outpainting.

Enhanced spatial coherence through circular padding.

Good scalability and generalization capabilities.

Abstract

Recent progress in panoramic image generation has underscored two critical limitations in existing approaches. First, most methods are built upon diffusion models, which are inherently ill-suited for equirectangular projection (ERP) panoramas due to the violation of the identically and independently distributed (i.i.d.) Gaussian noise assumption caused by their spherical mapping. Second, these methods often treat text-conditioned generation (text-to-panorama) and image-conditioned generation (panorama outpainting) as separate tasks, relying on distinct architectures and task-specific data. In this work, we propose a unified framework, Panoramic AutoRegressive model (PAR), which leverages masked autoregressive modeling to address these challenges. PAR avoids the i.i.d. assumption constraint and integrates text and image conditioning into a cohesive architecture, enabling seamless generation across tasks. To address the inherent discontinuity in existing generative models, we introduce circular padding to enhance spatial coherence and propose a consistency alignment strategy to improve generation quality. Extensive experiments demonstrate competitive performance in text-to-image generation and panorama outpainting tasks while showcasing promising scalability and generalization capabilities.