Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds

TL;DR

This paper extends the PointNet architecture to incorporate larger spatial context for 3D semantic segmentation of point clouds, leading to improved results on indoor and outdoor datasets.

Contribution

The authors propose two extensions to PointNet that enlarge the receptive field, enabling better use of spatial context in 3D point cloud segmentation.

Findings

Improved segmentation accuracy on indoor datasets

Enhanced outdoor scene segmentation performance

Effective incorporation of larger spatial context

Abstract

Deep learning approaches have made tremendous progress in the field of semantic segmentation over the past few years. However, most current approaches operate in the 2D image space. Direct semantic segmentation of unstructured 3D point clouds is still an open research problem. The recently proposed PointNet architecture presents an interesting step ahead in that it can operate on unstructured point clouds, achieving encouraging segmentation results. However, it subdivides the input points into a grid of blocks and processes each such block individually. In this paper, we investigate the question how such an architecture can be extended to incorporate larger-scale spatial context. We build upon PointNet and propose two extensions that enlarge the receptive field over the 3D scene. We evaluate the proposed strategies on challenging indoor and outdoor datasets and show improved results in both scenarios.