Learning Local Displacements for Point Cloud Completion

TL;DR

This paper introduces a novel neural network architecture with three specialized layers for 3D point cloud completion, achieving state-of-the-art results in object and scene reconstruction tasks.

Contribution

It presents a new encoder-decoder model with three innovative layers and a transformer-based extension for improved point cloud completion.

Findings

Achieves state-of-the-art performance on object completion tasks.

Demonstrates versatility with a transformer-based model.

Outperforms existing methods in indoor scene completion.

Abstract

We propose a novel approach aimed at object and semantic scene completion from a partial scan represented as a 3D point cloud. Our architecture relies on three novel layers that are used successively within an encoder-decoder structure and specifically developed for the task at hand. The first one carries out feature extraction by matching the point features to a set of pre-trained local descriptors. Then, to avoid losing individual descriptors as part of standard operations such as max-pooling, we propose an alternative neighbor-pooling operation that relies on adopting the feature vectors with the highest activations. Finally, up-sampling in the decoder modifies our feature extraction in order to increase the output dimension. While this model is already able to achieve competitive results with the state of the art, we further propose a way to increase the versatility of our approach to process point clouds. To this aim, we introduce a second model that assembles our layers within a transformer architecture. We evaluate both architectures on object and indoor scene completion tasks, achieving state-of-the-art performance.