MaFreeI2P: A Matching-Free Image-to-Point Cloud Registration Paradigm with Active Camera Pose Retrieval

TL;DR

MaFreeI2P introduces a novel matching-free approach for image-to-point cloud registration by actively retrieving camera pose through geometric feature contrast, avoiding information loss from traditional correspondence methods.

Contribution

The paper proposes a matching-free paradigm that uses cost volume and active pose retrieval, significantly improving registration accuracy and robustness over existing matching-based methods.

Findings

Achieves competitive accuracy on KITTI-Odometry and Apollo-DaoxiangLake datasets.

Effectively preserves information by using cost volume instead of correspondences.

Iterative pose sampling and filtering enhance convergence and precision.

Abstract

Image-to-point cloud registration seeks to estimate their relative camera pose, which remains an open question due to the data modality gaps. The recent matching-based methods tend to tackle this by building 2D-3D correspondences. In this paper, we reveal the information loss inherent in these methods and propose a matching-free paradigm, named MaFreeI2P. Our key insight is to actively retrieve the camera pose in SE(3) space by contrasting the geometric features between the point cloud and the query image. To achieve this, we first sample a set of candidate camera poses and construct their cost volume using the cross-modal features. Superior to matching, cost volume can preserve more information and its feature similarity implicitly reflects the confidence level of the sampled poses. Afterwards, we employ a convolutional network to adaptively formulate a similarity assessment function, where the input cost volume is further improved by filtering and pose-based weighting. Finally, we update the camera pose based on the similarity scores, and adopt a heuristic strategy to iteratively shrink the pose sampling space for convergence. Our MaFreeI2P achieves a very competitive registration accuracy and recall on the KITTI-Odometry and Apollo-DaoxiangLake datasets.