CMHANet: A Cross-Modal Hybrid Attention Network for Point Cloud Registration

TL;DR

CMHANet is a novel cross-modal attention network that fuses 2D image and 3D point cloud data, improving robustness and accuracy in point cloud registration under challenging real-world conditions.

Contribution

The paper introduces CMHANet, a new hybrid attention network that combines 2D images and 3D point clouds with a contrastive learning-based optimization for enhanced registration.

Findings

Outperforms existing methods on 3DMatch and 3DLoMatch datasets.

Demonstrates robustness to noise and partial data.

Shows strong generalization in zero-shot evaluations.

Abstract

Robust point cloud registration is a fundamental task in 3D computer vision and geometric deep learning, essential for applications such as large-scale 3D reconstruction, augmented reality, and scene understanding. However, the performance of established learning-based methods often degrades in complex, real world scenarios characterized by incomplete data, sensor noise, and low overlap regions. To address these limitations, we propose CMHANet, a novel Cross-Modal Hybrid Attention Network. Our method integrates the fusion of rich contextual information from 2D images with the geometric detail of 3D point clouds, yielding a comprehensive and resilient feature representation. Furthermore, we introduce an innovative optimization function based on contrastive learning, which enforces geometric consistency and significantly improves the model's robustness to noise and partial observations. We evaluated CMHANet on the 3DMatch and the challenging 3DLoMatch datasets. \rev{Additionally, zero-shot evaluations on the TUM RGB-D SLAM dataset verify the model's generalization capability to unseen domains.} The experimental results demonstrate that our method achieves substantial improvements in both registration accuracy and overall robustness, outperforming current techniques. We also release our code in \href{https://github.com/DongXu-Zhang/CMHANet}{https://github.com/DongXu-Zhang/CMHANet}.