A re-calibration method for object detection with multi-modal alignment bias in autonomous driving

TL;DR

This paper introduces a re-calibration method to mitigate the impact of sensor calibration bias on multi-modal object detection in autonomous driving, improving robustness and detection accuracy under real-world conditions.

Contribution

The paper proposes a novel re-calibration model that adjusts sensor alignment dynamically, enhancing detection performance and robustness against calibration biases in multi-modal autonomous driving systems.

Findings

Calibration bias significantly reduces detection performance.

The re-calibration model improves robustness and accuracy.

Method integrates semantic segmentation and tailored loss functions.

Abstract

Multi-modal object detection in autonomous driving has achieved great breakthroughs due to the usage of fusing complementary information from different sensors. The calibration in fusion between sensors such as LiDAR and camera was always supposed to be precise in previous work. However, in reality, calibration matrices are fixed when the vehicles leave the factory, but mechanical vibration, road bumps, and data lags may cause calibration bias. As there is relatively limited research on the impact of calibration on fusion detection performance, multi-sensor detection methods with flexible calibration dependency have remained a key objective. In this paper, we systematically evaluate the sensitivity of the SOTA EPNet++ detection framework and prove that even slight bias on calibration can reduce the performance seriously. To address this vulnerability, we propose a re-calibration model to re-calibrate the misalignment in detection tasks. This model integrates LiDAR point cloud, camera image, and initial calibration matrix as inputs, generating re-calibrated bias through semantic segmentation guidance and a tailored loss function design. The re-calibration model can operate with existing detection algorithms, enhancing both robustness against calibration bias and overall object detection performance. Our approach establishes a foundational methodology for maintaining reliability in multi-modal perception systems under real-world calibration uncertainties.