RLCNet: An end-to-end deep learning framework for simultaneous online calibration of LiDAR, RADAR, and Camera

TL;DR

RLCNet is a deep learning framework that enables real-time, simultaneous calibration of LiDAR, RADAR, and camera sensors, improving accuracy and robustness for autonomous vehicle perception in dynamic environments.

Contribution

The paper introduces RLCNet, an end-to-end trainable deep learning model for online calibration of multimodal sensors, with a novel online framework for dynamic adjustment and noise reduction.

Findings

Demonstrates superior calibration accuracy over existing methods.

Shows robustness under diverse real-world conditions.

Enables real-time calibration suitable for autonomous vehicles.

Abstract

Accurate extrinsic calibration of LiDAR, RADAR, and camera sensors is essential for reliable perception in autonomous vehicles. Still, it remains challenging due to factors such as mechanical vibrations and cumulative sensor drift in dynamic environments. This paper presents RLCNet, a novel end-to-end trainable deep learning framework for the simultaneous online calibration of these multimodal sensors. Validated on real-world datasets, RLCNet is designed for practical deployment and demonstrates robust performance under diverse conditions. To support real-time operation, an online calibration framework is introduced that incorporates a weighted moving average and outlier rejection, enabling dynamic adjustment of calibration parameters with reduced prediction noise and improved resilience to drift. An ablation study highlights the significance of architectural choices, while comparisons with existing methods demonstrate the superior accuracy and robustness of the proposed approach.