CLRNet: Targetless Extrinsic Calibration for Camera, Lidar and 4D Radar Using Deep Learning

TL;DR

CLRNet is a deep learning-based calibration method that accurately aligns camera, lidar, and radar sensors, outperforming existing methods and demonstrating strong domain transfer capabilities across datasets.

Contribution

The paper introduces CLRNet, a novel end-to-end deep learning framework for joint and pairwise calibration of camera, lidar, and radar sensors, incorporating multi-modal data and loop closure loss.

Findings

Reduces median translational and rotational errors by at least 50%.

Demonstrates superior calibration accuracy over state-of-the-art methods.

Shows effective domain transfer across different datasets.

Abstract

In this paper, we address extrinsic calibration for camera, lidar, and 4D radar sensors. Accurate extrinsic calibration of radar remains a challenge due to the sparsity of its data. We propose CLRNet, a novel, multi-modal end-to-end deep learning (DL) calibration network capable of addressing joint camera-lidar-radar calibration, or pairwise calibration between any two of these sensors. We incorporate equirectangular projection, camera-based depth image prediction, additional radar channels, and leverage lidar with a shared feature space and loop closure loss. In extensive experiments using the View-of-Delft and Dual-Radar datasets, we demonstrate superior calibration accuracy compared to existing state-of-the-art methods, reducing both median translational and rotational calibration errors by at least 50%. Finally, we examine the domain transfer capabilities of the proposed network and baselines, when evaluating across datasets. The code will be made publicly available upon acceptance at: https://github.com/tudelft-iv.