Automatic Target-Less Camera-LiDAR Calibration From Motion and Deep Point Correspondences

TL;DR

This paper introduces MDPCalib, a target-less, deep learning-based method for automatic camera-LiDAR calibration that leverages sensor motion and point correspondences, applicable across diverse robotic platforms without human supervision.

Contribution

The novel approach eliminates the need for target objects and in-domain retraining, enabling accurate and robust calibration using motion estimates and deep correspondences.

Findings

Achieves highly accurate extrinsic calibration parameters.

Demonstrates robustness to random initialization.

Generalizes across various robotic platforms.

Abstract

Sensor setups of robotic platforms commonly include both camera and LiDAR as they provide complementary information. However, fusing these two modalities typically requires a highly accurate calibration between them. In this paper, we propose MDPCalib which is a novel method for camera-LiDAR calibration that requires neither human supervision nor any specific target objects. Instead, we utilize sensor motion estimates from visual and LiDAR odometry as well as deep learning-based 2D-pixel-to-3D-point correspondences that are obtained without in-domain retraining. We represent camera-LiDAR calibration as an optimization problem and minimize the costs induced by constraints from sensor motion and point correspondences. In extensive experiments, we demonstrate that our approach yields highly accurate extrinsic calibration parameters and is robust to random initialization. Additionally, our approach generalizes to a wide range of sensor setups, which we demonstrate by employing it on various robotic platforms including a self-driving perception car, a quadruped robot, and a UAV. To make our calibration method publicly accessible, we release the code on our project website at http://calibration.cs.uni-freiburg.de.