Boosting 3D Point Cloud Registration by Transferring Multi-modality Knowledge

TL;DR

This paper introduces a novel approach to enhance 3D point cloud registration by transferring knowledge from pre-trained multi-modality models, improving accuracy without requiring multiple modalities during inference.

Contribution

It proposes a new ensemble descriptor neural network that leverages multi-modality knowledge transfer, achieving state-of-the-art results with only point clouds at inference.

Findings

Achieves state-of-the-art results on 3DMatch.

Demonstrates competitive accuracy on 3DLoMatch and KITTI.

Enhances registration accuracy by knowledge transfer.

Abstract

The recent multi-modality models have achieved great performance in many vision tasks because the extracted features contain the multi-modality knowledge. However, most of the current registration descriptors have only concentrated on local geometric structures. This paper proposes a method to boost point cloud registration accuracy by transferring the multi-modality knowledge of pre-trained multi-modality model to a new descriptor neural network. Different to the previous multi-modality methods that requires both modalities, the proposed method only requires point clouds during inference. Specifically, we propose an ensemble descriptor neural network combining pre-trained sparse convolution branch and a new point-based convolution branch. By fine-tuning on a single modality data, the proposed method achieves new state-of-the-art results on 3DMatch and competitive accuracy on 3DLoMatch and KITTI.