PlueckerNet: Learn to Register 3D Line Reconstructions

TL;DR

PlueckerNet is a neural network that learns to accurately register 3D line reconstructions by jointly estimating correspondences and relative pose using a novel Pluecker representation and optimal transport-based matching.

Contribution

The paper introduces a new neural network architecture combining Pluecker line representations, optimal transport, and RANSAC for improved 3D line registration accuracy.

Findings

Outperforms baseline methods in registration precision

Effective on both indoor and outdoor datasets

Accurately estimates 6-DoF transformations

Abstract

Aligning two partially-overlapped 3D line reconstructions in Euclidean space is challenging, as we need to simultaneously solve correspondences and relative pose between line reconstructions. This paper proposes a neural network based method and it has three modules connected in sequence: (i) a Multilayer Perceptron (MLP) based network takes Pluecker representations of lines as inputs, to extract discriminative line-wise features and matchabilities (how likely each line is going to have a match), (ii) an Optimal Transport (OT) layer takes two-view line-wise features and matchabilities as inputs to estimate a 2D joint probability matrix, with each item describes the matchness of a line pair, and (iii) line pairs with Top-K matching probabilities are fed to a 2-line minimal solver in a RANSAC framework to estimate a six Degree-of-Freedom (6-DoF) rigid transformation. Experiments on both indoor and outdoor datasets show that the registration (rotation and translation) precision of our method outperforms baselines significantly.