PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

TL;DR

PointNet introduces a neural network architecture that directly processes raw 3D point clouds, achieving high efficiency and accuracy in classification and segmentation tasks while respecting the permutation invariance of input points.

Contribution

The paper presents a novel neural network, PointNet, that directly consumes point clouds without voxelization, maintaining permutation invariance and enabling effective 3D data analysis.

Findings

Achieves state-of-the-art performance on 3D classification and segmentation tasks.

Demonstrates robustness to input perturbations and noise.

Provides theoretical analysis of learned features and network robustness.

Abstract

Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds and well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective. Empirically, it shows strong performance on par or even better than state of the art. Theoretically, we provide analysis towards understanding of what the network has learnt and why the network is robust with respect to input perturbation and corruption.