End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds

TL;DR

This paper introduces an end-to-end neural framework for learning local multi-view descriptors of 3D point clouds, integrating differentiable rendering and soft-view pooling to improve registration accuracy.

Contribution

It presents a novel differentiable rendering approach with learnable viewpoints and a soft-view pooling module for more effective local descriptor learning.

Findings

Outperforms existing local descriptors on 3D registration benchmarks

Achieves higher discriminative power and robustness

Demonstrates the effectiveness of end-to-end learning with differentiable rendering

Abstract

In this work, we propose an end-to-end framework to learn local multi-view descriptors for 3D point clouds. To adopt a similar multi-view representation, existing studies use hand-crafted viewpoints for rendering in a preprocessing stage, which is detached from the subsequent descriptor learning stage. In our framework, we integrate the multi-view rendering into neural networks by using a differentiable renderer, which allows the viewpoints to be optimizable parameters for capturing more informative local context of interest points. To obtain discriminative descriptors, we also design a soft-view pooling module to attentively fuse convolutional features across views. Extensive experiments on existing 3D registration benchmarks show that our method outperforms existing local descriptors both quantitatively and qualitatively.