CorrI2P: Deep Image-to-Point Cloud Registration via Dense Correspondence

TL;DR

CorrI2P introduces a novel dense correspondence framework for accurate image-to-point cloud registration by identifying overlapping regions and establishing 2D-3D correspondences, significantly outperforming existing methods.

Contribution

First feature-based dense correspondence framework for image-to-point cloud registration, integrating region detection and pose estimation.

Findings

Outperforms state-of-the-art methods on KITTI and NuScenes datasets

Effective in identifying overlapping regions for registration

Demonstrates robustness and accuracy in 2D-3D alignment

Abstract

Motivated by the intuition that the critical step of localizing a 2D image in the corresponding 3D point cloud is establishing 2D-3D correspondence between them, we propose the first feature-based dense correspondence framework for addressing the image-to-point cloud registration problem, dubbed CorrI2P, which consists of three modules, i.e., feature embedding, symmetric overlapping region detection, and pose estimation through the established correspondence. Specifically, given a pair of a 2D image and a 3D point cloud, we first transform them into high-dimensional feature space and feed the resulting features into a symmetric overlapping region detector to determine the region where the image and point cloud overlap each other. Then we use the features of the overlapping regions to establish the 2D-3D correspondence before running EPnP within RANSAC to estimate the camera's pose. Experimental results on KITTI and NuScenes datasets show that our CorrI2P outperforms state-of-the-art image-to-point cloud registration methods significantly. We will make the code publicly available.