RaLF: Flow-based Global and Metric Radar Localization in LiDAR Maps

TL;DR

RaLF is a deep learning approach that robustly localizes radar scans within LiDAR maps, combining place recognition and metric localization to improve autonomous robot navigation under challenging conditions.

Contribution

Introduces RaLF, a novel neural network that jointly learns place recognition and metric localization for radar-LiDAR integration, demonstrating state-of-the-art results and generalization capabilities.

Findings

RaLF achieves state-of-the-art performance in place recognition.

RaLF effectively predicts 3-DoF transformations for metric localization.

RaLF generalizes well across different cities and sensor setups.

Abstract

Localization is paramount for autonomous robots. While camera and LiDAR-based approaches have been extensively investigated, they are affected by adverse illumination and weather conditions. Therefore, radar sensors have recently gained attention due to their intrinsic robustness to such conditions. In this paper, we propose RaLF, a novel deep neural network-based approach for localizing radar scans in a LiDAR map of the environment, by jointly learning to address both place recognition and metric localization. RaLF is composed of radar and LiDAR feature encoders, a place recognition head that generates global descriptors, and a metric localization head that predicts the 3-DoF transformation between the radar scan and the map. We tackle the place recognition task by learning a shared embedding space between the two modalities via cross-modal metric learning. Additionally, we perform metric localization by predicting pixel-level flow vectors that align the query radar scan with the LiDAR map. We extensively evaluate our approach on multiple real-world driving datasets and show that RaLF achieves state-of-the-art performance for both place recognition and metric localization. Moreover, we demonstrate that our approach can effectively generalize to different cities and sensor setups than the ones used during training. We make the code and trained models publicly available at http://ralf.cs.uni-freiburg.de.