Pano360: Perspective to Panoramic Vision with Geometric Consistency

TL;DR

Pano360 introduces a 3D-aware, transformer-based panoramic stitching method that leverages geometric consistency across multiple views to improve alignment and reduce distortion in challenging scenes.

Contribution

The paper presents a novel 3D photogrammetric approach with a transformer architecture for panoramic stitching, enhancing accuracy and global consistency over traditional 2D methods.

Findings

Outperforms existing methods in alignment accuracy

Reduces distortion and misalignment in challenging scenes

Provides a large-scale real-world dataset for benchmarking

Abstract

Prior panorama stitching approaches heavily rely on pairwise feature correspondences and are unable to leverage geometric consistency across multiple views. This leads to severe distortion and misalignment, especially in challenging scenes with weak textures, large parallax, and repetitive patterns. Given that multi-view geometric correspondences can be directly constructed in 3D space, making them more accurate and globally consistent, we extend the 2D alignment task to the 3D photogrammetric space. We adopt a novel transformer-based architecture to achieve 3D awareness and aggregate global information across all views. It directly utilizes camera poses to guide image warping for global alignment in 3D space and employs a multi-feature joint optimization strategy to compute the seams. Additionally, to establish an evaluation benchmark and train our network, we constructed a large-scale dataset of real-world scenes. Extensive experiments show that our method significantly outperforms existing alternatives in alignment accuracy and perceptual quality.