One target to align them all: LiDAR, RGB and event cameras extrinsic calibration for Autonomous Driving

TL;DR

This paper introduces a unified multi-modal calibration method for autonomous driving sensors, using a specially designed 3D target to simultaneously calibrate LiDAR, RGB, and event cameras in a single step.

Contribution

The paper presents a novel 3D calibration target and a joint calibration framework enabling one-shot multi-sensor extrinsic calibration for LiDAR, RGB, and event cameras.

Findings

Accurate calibration demonstrated on a custom autonomous driving dataset.

Robustness confirmed across various sensor configurations.

Outperforms pairwise calibration methods in accuracy.

Abstract

We present a novel multi-modal extrinsic calibration framework designed to simultaneously estimate the relative poses between event cameras, LiDARs, and RGB cameras, with particular focus on the challenging event camera calibration. Core of our approach is a novel 3D calibration target, specifically designed and constructed to be concurrently perceived by all three sensing modalities. The target encodes features in planes, ChArUco, and active LED patterns, each tailored to the unique characteristics of LiDARs, RGB cameras, and event cameras respectively. This unique design enables a one-shot, joint extrinsic calibration process, in contrast to existing approaches that typically rely on separate, pairwise calibrations. Our calibration pipeline is designed to accurately calibrate complex vision systems in the context of autonomous driving, where precise multi-sensor alignment is critical. We validate our approach through an extensive experimental evaluation on a custom built dataset, recorded with an advanced autonomous driving sensor setup, confirming the accuracy and robustness of our method.