LiftProj: Space Lifting and Projection-Based Panorama Stitching

TL;DR

LiftProj introduces a 3D space lifting and projection framework for panoramic stitching, significantly reducing distortions and ghosting in complex scenes with multiple depth layers and occlusions, outperforming traditional 2D methods.

Contribution

This paper presents a novel 3D space lifting and projection approach for panorama stitching, moving beyond 2D warping to achieve more geometrically consistent results in complex scenes.

Findings

Reduces geometric distortions in challenging scenes

Mitigates ghosting artifacts effectively

Produces more natural and consistent panoramic images

Abstract

Traditional image stitching techniques have predominantly utilized two-dimensional homography transformations and mesh warping to achieve alignment on a planar surface. While effective for scenes that are approximately coplanar or exhibit minimal parallax, these approaches often result in ghosting, structural bending, and stretching distortions in non-overlapping regions when applied to real three-dimensional scenes characterized by multiple depth layers and occlusions. Such challenges are exacerbated in multi-view accumulations and 360{\deg} closed-loop stitching scenarios. In response, this study introduces a spatially lifted panoramic stitching framework that initially elevates each input image into a dense three-dimensional point representation within a unified coordinate system, facilitating global cross-view fusion augmented by confidence metrics. Subsequently, a unified projection center is established in three-dimensional space, and an equidistant cylindrical projection is employed to map the fused data onto a single panoramic manifold, thereby producing a geometrically consistent 360{\deg} panoramic layout. Finally, hole filling is conducted within the canvas domain to address unknown regions revealed by viewpoint transitions, restoring continuous texture and semantic coherence. This framework reconceptualizes stitching from a two-dimensional warping paradigm to a three-dimensional consistency paradigm and is designed to flexibly incorporate various three-dimensional lifting and completion modules. Experimental evaluations demonstrate that the proposed method substantially mitigates geometric distortions and ghosting artifacts in scenarios involving significant parallax and complex occlusions, yielding panoramic results that are more natural and consistent.