Robust place recognition under illumination changes using pseudo-LiDAR from omnidirectional images

TL;DR

This paper introduces a new method for visual place recognition that is robust to changes in lighting by using pseudo-LiDAR data from depth maps.

Contribution

The novel framework uses depth estimation and pseudo-LiDAR to improve VPR robustness under illumination changes and different camera types.

Findings

The system performs well across diverse lighting conditions despite being trained only under cloudy conditions.

The approach generalizes to geometrically dissimilar inputs and different camera types.

Pseudo-LiDAR from standard cameras offers a cost-effective alternative to 3D sensors for VPR.

Abstract

Visual Place Recognition (VPR) systems typically exhibit reduced robustness when subjected to changes in scene appearance produced by illumination dynamics or heterogeneity across different types of visual sensors. This paper proposes a novel framework that exploits depth estimation techniques to overcome these challenges. Our approach transforms omnidirectional images into depth maps using Distill Any Depth, a state-of-the-art depth estimator based on Depth Anything V2. These depth maps are then converted into pseudo-LiDAR point clouds, which serve as input to the MinkUNeXt architecture, which generates global-appearance descriptors. A key innovation lies in our novel data augmentation technique that exploits different distilled variants of depth estimation models to enhance robustness across varying conditions. Despite training with a limited set of images captured only under cloudy conditions, our system demonstrates robust performance when evaluated across diverse lighting scenarios, and further tests with different datasets and camera types confirm its generalization to geometrically dissimilar inputs. Extensive comparisons with state-of-the-art methods prove that our approach performs competitively across diverse lighting conditions, particularly excelling in scenarios with significant illumination changes. Furthermore, the generation of pseudo-LiDAR information from standard cameras provides a cost-effective alternative to 3D sensors. In summary, this work presents a fundamentally different approach to scene representation for VPR, with promising implications for robot localization in challenging environments. The implementation is publicly available at https://juanjo-cabrera.github.io/projects-pL-MinkUNeXt/.

Figures