CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks

TL;DR

CalibDNN is a fully automatic deep learning method for calibrating multimodal sensors like LiDAR and cameras without special targets, achieving state-of-the-art accuracy through extensive experiments.

Contribution

It introduces a novel deep neural network approach that calibrates multimodal sensors automatically and efficiently without requiring calibration targets or hardware aids.

Findings

Achieves state-of-the-art calibration accuracy.

Operates fully automatically with a single model.

Validated on multiple datasets with extensive experiments.

Abstract

Current perception systems often carry multimodal imagers and sensors such as 2D cameras and 3D LiDAR sensors. To fuse and utilize the data for downstream perception tasks, robust and accurate calibration of the multimodal sensor data is essential. We propose a novel deep learning-driven technique (CalibDNN) for accurate calibration among multimodal sensor, specifically LiDAR-Camera pairs. The key innovation of the proposed work is that it does not require any specific calibration targets or hardware assistants, and the entire processing is fully automatic with a single model and single iteration. Results comparison among different methods and extensive experiments on different datasets demonstrates the state-of-the-art performance.