Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks

TL;DR

This paper introduces a versatile deep learning building block for point cloud processing that effectively handles various tasks by combining spatial transformers, multi-view convolution, and dense grids, achieving state-of-the-art results.

Contribution

A novel, multi-head differentiable rasterization block that enhances deep point cloud architectures for diverse discriminative and generative tasks.

Findings

Achieves state-of-the-art performance in point cloud segmentation and classification.

Demonstrates versatility across multiple point cloud processing tasks.

Improves efficiency by integrating dense convolution with multi-view rasterization.

Abstract

We present a new versatile building block for deep point cloud processing architectures that is equally suited for diverse tasks. This building block combines the ideas of spatial transformers and multi-view convolutional networks with the efficiency of standard convolutional layers in two and three-dimensional dense grids. The new block operates via multiple parallel heads, whereas each head differentiably rasterizes feature representations of individual points into a low-dimensional space, and then uses dense convolution to propagate information across points. The results of the processing of individual heads are then combined together resulting in the update of point features. Using the new block, we build architectures for both discriminative (point cloud segmentation, point cloud classification) and generative (point cloud inpainting and image-based point cloud reconstruction) tasks. The resulting architectures achieve state-of-the-art performance for these tasks, demonstrating the versatility of the new block for point cloud processing.