Quantity-Aware Coarse-to-Fine Correspondence for Image-to-Point Cloud Registration

TL;DR

This paper introduces a quantity-aware coarse-to-fine framework for image-to-point cloud registration, improving matching accuracy by leveraging semantic alignment and adaptive correlation quantification, leading to superior performance on benchmark datasets.

Contribution

It proposes a novel quantity-aware correspondence framework with adaptive supervision and refinement strategies for improved image-to-point cloud registration.

Findings

Outperforms state-of-the-art methods on KITTI Odometry dataset.

Achieves higher registration accuracy on NuScenes dataset.

Demonstrates robustness and efficiency in correspondence matching.

Abstract

Image-to-point cloud registration aims to determine the relative camera pose between an RGB image and a reference point cloud, serving as a general solution for locating 3D objects from 2D observations. Matching individual points with pixels can be inherently ambiguous due to modality gaps. To address this challenge, we propose a framework to capture quantity-aware correspondences between local point sets and pixel patches and refine the results at both the point and pixel levels. This framework aligns the high-level semantics of point sets and pixel patches to improve the matching accuracy. On a coarse scale, the set-to-patch correspondence is expected to be influenced by the quantity of 3D points. To achieve this, a novel supervision strategy is proposed to adaptively quantify the degrees of correlation as continuous values. On a finer scale, point-to-pixel correspondences are refined from a smaller search space through a well-designed scheme, which incorporates both resampling and quantity-aware priors. Particularly, a confidence sorting strategy is proposed to proportionally select better correspondences at the final stage. Leveraging the advantages of high-quality correspondences, the problem is successfully resolved using an efficient Perspective-n-Point solver within the framework of random sample consensus (RANSAC). Extensive experiments on the KITTI Odometry and NuScenes datasets demonstrate the superiority of our method over the state-of-the-art methods.