Deep Robust Multi-Robot Re-localisation in Natural Environments

TL;DR

This paper introduces a deep learning-based method for robust multi-robot re-localisation in natural environments, using lidar-image cross-modality to improve place recognition and localisation accuracy in challenging scenarios.

Contribution

It presents a novel self-supervised deep network that combines lidar and image data for improved re-localisation in unstructured environments, addressing limitations of single-modality approaches.

Findings

Effective lidar-image cross-modality re-localisation demonstrated on real datasets.

Improved robustness in unstructured natural environments.

Model accurately predicts alignment and misalignment in real-time.

Abstract

The success of re-localisation has crucial implications for the practical deployment of robots operating within a prior map or relative to one another in real-world scenarios. Using single-modality, place recognition and localisation can be compromised in challenging environments such as forests. To address this, we propose a strategy to prevent lidar-based re-localisation failure using lidar-image cross-modality. Our solution relies on self-supervised 2D-3D feature matching to predict alignment and misalignment. Leveraging a deep network for lidar feature extraction and relative pose estimation between point clouds, we train a model to evaluate the estimated transformation. A model predicting the presence of misalignment is learned by analysing image-lidar similarity in the embedding space and the geometric constraints available within the region seen in both modalities in Euclidean space. Experimental results using real datasets (offline and online modes) demonstrate the effectiveness of the proposed pipeline for robust re-localisation in unstructured, natural environments.