GLASS: Geometry-aware Local Alignment and Structure Synchronization Network for 2D-3D Registration

TL;DR

This paper introduces a geometry-aware network with novel modules to improve 2D-3D registration accuracy, especially in scenes with repetitive patterns, by enhancing features with geometric cues and enforcing structural consistency.

Contribution

The paper proposes the LGE and GDC modules that incorporate geometric structure and distribution constraints to improve registration performance over prior methods.

Findings

Achieves state-of-the-art results on RGB-D Scenes v2 and 7-Scenes benchmarks.

Enhances feature robustness by injecting geometric cues with normal vectors.

Effectively constrains feature similarity distributions for better alignment.

Abstract

Image-to-point cloud registration methods typically follow a coarse-to-fine pipeline, extracting patch-level correspondences and refining them into dense pixel-to-point matches. However, in scenes with repetitive patterns, images often lack sufficient 3D structural cues and alignment with point clouds, leading to incorrect matches. Moreover, prior methods usually overlook structural consistency, limiting the full exploitation of correspondences. To address these issues, we propose two novel modules: the Local Geometry Enhancement (LGE) module and the Graph Distribution Consistency (GDC) module. LGE enhances both image and point cloud features with normal vectors, injecting geometric structure into image features to reduce mismatches. GDC constructs a graph from matched points to update features and explicitly constrain similarity distributions. Extensive experiments and ablations on two benchmarks, RGB-D Scenes v2 and 7-Scenes, demonstrate that our approach achieves state-of-the-art performance in image-to-point cloud registration.