What Stops Learning-based 3D Registration from Working in the Real World?

TL;DR

This paper investigates why learning-based 3D registration methods fail in real-world scenarios, identifies key issues, proposes guidelines to improve training and testing, and introduces BPNet, a model that generalizes well to real data without fine-tuning.

Contribution

The paper identifies failure sources in real-world 3D registration, proposes effective guidelines, and develops BPNet, the first learning-based method to handle unseen objects in real-world data without fine-tuning.

Findings

Guidelines improve training convergence and accuracy of baseline methods

BPNet achieves up to 67% accuracy on real-world unseen objects

Model trained only on synthetic data generalizes effectively to real data

Abstract

Much progress has been made on the task of learning-based 3D point cloud registration, with existing methods yielding outstanding results on standard benchmarks, such as ModelNet40, even in the partial-to-partial matching scenario. Unfortunately, these methods still struggle in the presence of real data. In this work, we identify the sources of these failures, analyze the reasons behind them, and propose solutions to tackle them. We summarise our findings into a set of guidelines and demonstrate their effectiveness by applying them to different baseline methods, DCP and IDAM. In short, our guidelines improve both their training convergence and testing accuracy. Ultimately, this translates to a best-practice 3D registration network (BPNet), constituting the first learning-based method able to handle previously-unseen objects in real-world data. Despite being trained only on synthetic data, our model generalizes to real data without any fine-tuning, reaching an accuracy of up to 67% on point clouds of unseen objects obtained with a commercial sensor.