MortonNet: Self-Supervised Learning of Local Features in 3D Point Clouds

TL;DR

MortonNet introduces a self-supervised learning approach for point-wise feature extraction in 3D point clouds, improving segmentation performance and generalizing across datasets by predicting the next point in a Morton-order sequence.

Contribution

The paper proposes a novel self-supervised task using RNNs on Morton-order sequences to learn local features in point clouds, enhancing 3D segmentation tasks.

Findings

Improves semantic segmentation accuracy by 3% on S3DIS.

Transfers well to vKITTI, outperforming state-of-the-art by 3.8%.

Enables simpler models for part segmentation with stable training.

Abstract

We present a self-supervised task on point clouds, in order to learn meaningful point-wise features that encode local structure around each point. Our self-supervised network, named MortonNet, operates directly on unstructured/unordered point clouds. Using a multi-layer RNN, MortonNet predicts the next point in a point sequence created by a popular and fast Space Filling Curve, the Morton-order curve. The final RNN state (coined Morton feature) is versatile and can be used in generic 3D tasks on point clouds. In fact, we show how Morton features can be used to significantly improve performance (+3% for 2 popular semantic segmentation algorithms) in the task of semantic segmentation of point clouds on the challenging and large-scale S3DIS dataset. We also show how MortonNet trained on S3DIS transfers well to another large-scale dataset, vKITTI, leading to an improvement over state-of-the-art of 3.8%. Finally, we use Morton features to train a much simpler and more stable model for part segmentation in ShapeNet. Our results show how our self-supervised task results in features that are useful for 3D segmentation tasks, and generalize well to other datasets.