Adaptive Agent Selection and Interaction Network for Image-to-point cloud Registration

TL;DR

This paper introduces a novel cross-modal registration framework with adaptive agent selection and interaction modules, significantly improving robustness and accuracy in image-to-point cloud registration under noisy conditions.

Contribution

It proposes the IAS and RAI modules that enhance feature selection and interaction, leading to state-of-the-art results in challenging registration scenarios.

Findings

Achieves state-of-the-art performance on RGB-D Scenes v2 and 7-Scenes benchmarks.

Improves robustness and accuracy in noisy and challenging registration conditions.

Effectively reduces mismatches through reliable agent interaction.

Abstract

Typical detection-free methods for image-to-point cloud registration leverage transformer-based architectures to aggregate cross-modal features and establish correspondences. However, they often struggle under challenging conditions, where noise disrupts similarity computation and leads to incorrect correspondences. Moreover, without dedicated designs, it remains difficult to effectively select informative and correlated representations across modalities, thereby limiting the robustness and accuracy of registration. To address these challenges, we propose a novel cross-modal registration framework composed of two key modules: the Iterative Agents Selection (IAS) module and the Reliable Agents Interaction (RAI) module. IAS enhances structural feature awareness with phase maps and employs reinforcement learning principles to efficiently select reliable agents. RAI then leverages these selected agents to guide cross-modal interactions, effectively reducing mismatches and improving overall robustness. Extensive experiments on the RGB-D Scenes v2 and 7-Scenes benchmarks demonstrate that our method consistently achieves state-of-the-art performance.