Interpolated Convolutional Networks for 3D Point Cloud Understanding

TL;DR

This paper introduces Interpolated Convolution (InterpConv), a novel convolution operation for 3D point clouds that handles irregular, sparse data directly, enabling effective shape classification and segmentation.

Contribution

The paper proposes InterpConv, a permutation and sparsity invariant convolution method for point clouds, and demonstrates its effectiveness in various recognition tasks.

Findings

Achieves state-of-the-art results on ModelNet40, ShapeNet Parts, and S3DIS datasets.

Effectively captures local and global features in point cloud data.

Handles irregular, sparse, and unordered point cloud inputs directly.

Abstract

Point cloud is an important type of 3D representation. However, directly applying convolutions on point clouds is challenging due to the sparse, irregular and unordered data structure. In this paper, we propose a novel Interpolated Convolution operation, InterpConv, to tackle the point cloud feature learning and understanding problem. The key idea is to utilize a set of discrete kernel weights and interpolate point features to neighboring kernel-weight coordinates by an interpolation function for convolution. A normalization term is introduced to handle neighborhoods of different sparsity levels. Our InterpConv is shown to be permutation and sparsity invariant, and can directly handle irregular inputs. We further design Interpolated Convolutional Neural Networks (InterpCNNs) based on InterpConv layers to handle point cloud recognition tasks including shape classification, object part segmentation and indoor scene semantic parsing. Experiments show that the networks can capture both fine-grained local structures and global shape context information effectively. The proposed approach achieves state-of-the-art performance on public benchmarks including ModelNet40, ShapeNet Parts and S3DIS.