CFNet: LiDAR-Camera Registration Using Calibration Flow Network

TL;DR

This paper introduces CFNet, an innovative online calibration method combining deep learning and geometric algorithms to improve LiDAR-camera extrinsic calibration accuracy and robustness, demonstrated on KITTI datasets.

Contribution

The paper presents a novel calibration flow network and a semantic initialization algorithm that outperform existing methods in LiDAR-camera calibration tasks.

Findings

Superior accuracy on KITTI datasets

Effective combination of deep learning and geometric methods

Robustness to different calibration scenarios

Abstract

As an essential procedure of data fusion, LiDAR-camera calibration is critical for autonomous vehicles and robot navigation. Most calibration methods rely on hand-crafted features and require significant amounts of extracted features or specific calibration targets. With the development of deep learning (DL) techniques, some attempts take advantage of convolutional neural networks (CNNs) to regress the 6 degrees of freedom (DOF) extrinsic parameters. Nevertheless, the performance of these DL-based methods is reported to be worse than the non-DL methods. This paper proposed an online LiDAR-camera extrinsic calibration algorithm that combines the DL and the geometry methods. We define a two-channel image named calibration flow to illustrate the deviation from the initial projection to the ground truth. EPnP algorithm within the RANdom SAmple Consensus (RANSAC) scheme is applied to estimate the extrinsic parameters with 2D-3D correspondences constructed by the calibration flow. Experiments on KITTI datasets demonstrate that our proposed method is superior to the state-of-the-art methods. Furthermore, we propose a semantic initialization algorithm with the introduction of instance centroids (ICs). The code will be publicly available at https://github.com/LvXudong-HIT/CFNet.