CPO: Change Robust Panorama to Point Cloud Localization

TL;DR

CPO is a fast, robust panorama-to-point cloud localization algorithm that handles scene changes effectively by using color histograms and score maps, avoiding traditional feature matching.

Contribution

It introduces a novel approach leveraging spherical projection equivariance and color histograms for robust localization amidst scene changes.

Findings

Achieves stable localization despite scene changes and repetitive structures

Operates efficiently without explicit image rendering for all poses

Demonstrates effectiveness across various challenging scenarios

Abstract

We present CPO, a fast and robust algorithm that localizes a 2D panorama with respect to a 3D point cloud of a scene possibly containing changes. To robustly handle scene changes, our approach deviates from conventional feature point matching, and focuses on the spatial context provided from panorama images. Specifically, we propose efficient color histogram generation and subsequent robust localization using score maps. By utilizing the unique equivariance of spherical projections, we propose very fast color histogram generation for a large number of camera poses without explicitly rendering images for all candidate poses. We accumulate the regional consistency of the panorama and point cloud as 2D/3D score maps, and use them to weigh the input color values to further increase robustness. The weighted color distribution quickly finds good initial poses and achieves stable convergence for gradient-based optimization. CPO is lightweight and achieves effective localization in all tested scenarios, showing stable performance despite scene changes, repetitive structures, or featureless regions, which are typical challenges for visual localization with perspective cameras. Code is available at \url{https://github.com/82magnolia/panoramic-localization/}.