Joint Intrinsic and Extrinsic LiDAR-Camera Calibration in Targetless Environments Using Plane-Constrained Bundle Adjustment

TL;DR

This paper presents a novel targetless calibration method for LiDAR-camera systems that jointly optimizes intrinsic and extrinsic parameters using plane-constrained bundle adjustment, improving accuracy in challenging environments.

Contribution

The method uniquely integrates visual bundle adjustment with LiDAR point cloud plane registration in a unified optimization framework for joint calibration.

Findings

Outperforms state-of-the-art calibration methods.

Maintains high accuracy in challenging environments.

Open-sourced code for community use.

Abstract

This paper introduces a novel targetless method for joint intrinsic and extrinsic calibration of LiDAR-camera systems using plane-constrained bundle adjustment (BA). Our method leverages LiDAR point cloud measurements from planes in the scene, alongside visual points derived from those planes. The core novelty of our method lies in the integration of visual BA with the registration between visual points and LiDAR point cloud planes, which is formulated as a unified optimization problem. This formulation achieves concurrent intrinsic and extrinsic calibration, while also imparting depth constraints to the visual points to enhance the accuracy of intrinsic calibration. Experiments are conducted on both public data sequences and self-collected dataset. The results showcase that our approach not only surpasses other state-of-the-art (SOTA) methods but also maintains remarkable calibration accuracy even within challenging environments. For the benefits of the robotics community, we have open sourced our codes.