PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registration

TL;DR

PointMBF introduces a multi-scale bidirectional fusion network that effectively combines RGB and depth data for unsupervised point cloud registration, achieving state-of-the-art results on benchmark datasets.

Contribution

The paper proposes a novel multi-scale bidirectional fusion approach for unsupervised RGB-D point cloud registration, improving feature extraction and registration accuracy.

Findings

Achieves new state-of-the-art performance on ScanNet and 3DMatch datasets.

Effectively leverages complementary RGB and depth information through bidirectional fusion.

Demonstrates significant improvement over existing unsupervised registration methods.

Abstract

Point cloud registration is a task to estimate the rigid transformation between two unaligned scans, which plays an important role in many computer vision applications. Previous learning-based works commonly focus on supervised registration, which have limitations in practice. Recently, with the advance of inexpensive RGB-D sensors, several learning-based works utilize RGB-D data to achieve unsupervised registration. However, most of existing unsupervised methods follow a cascaded design or fuse RGB-D data in a unidirectional manner, which do not fully exploit the complementary information in the RGB-D data. To leverage the complementary information more effectively, we propose a network implementing multi-scale bidirectional fusion between RGB images and point clouds generated from depth images. By bidirectionally fusing visual and geometric features in multi-scales, more distinctive deep features for correspondence estimation can be obtained, making our registration more accurate. Extensive experiments on ScanNet and 3DMatch demonstrate that our method achieves new state-of-the-art performance. Code will be released at https://github.com/phdymz/PointMBF