Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

TL;DR

This paper introduces a deep learning framework for long-term place recognition that effectively matches radar scans to lidar maps, enabling robust localization across different sensors and conditions.

Contribution

It presents a joint training neural network that learns shared embeddings for radar and lidar, allowing heterogeneous place recognition with a single training process.

Findings

Effective radar-to-lidar place recognition demonstrated

Model generalizes well across multiple sessions and conditions

Supports lidar-to-lidar, radar-to-radar, and radar-to-lidar matching

Abstract

Place recognition is critical for both offline mapping and online localization. However, current single-sensor based place recognition still remains challenging in adverse conditions. In this paper, a heterogeneous measurements based framework is proposed for long-term place recognition, which retrieves the query radar scans from the existing lidar maps. To achieve this, a deep neural network is built with joint training in the learning stage, and then in the testing stage, shared embeddings of radar and lidar are extracted for heterogeneous place recognition. To validate the effectiveness of the proposed method, we conduct tests and generalization experiments on the multi-session public datasets compared to other competitive methods. The experimental results indicate that our model is able to perform multiple place recognitions: lidar-to-lidar, radar-to-radar and radar-to-lidar, while the learned model is trained only once. We also release the source code publicly: https://github.com/ZJUYH/radar-to-lidar-place-recognition.