Point Transformer

TL;DR

Point Transformer is a neural network designed for point cloud data that captures local and global features using local-global attention, permutation invariance, and achieves competitive results on classification and segmentation tasks.

Contribution

It introduces the Point Transformer architecture with local-global attention and SortNet for permutation invariance, advancing point cloud processing methods.

Findings

Achieves competitive classification accuracy.

Effective part segmentation performance.

Permutation invariance via SortNet.

Abstract

In this work, we present Point Transformer, a deep neural network that operates directly on unordered and unstructured point sets. We design Point Transformer to extract local and global features and relate both representations by introducing the local-global attention mechanism, which aims to capture spatial point relations and shape information. For that purpose, we propose SortNet, as part of the Point Transformer, which induces input permutation invariance by selecting points based on a learned score. The output of Point Transformer is a sorted and permutation invariant feature list that can directly be incorporated into common computer vision applications. We evaluate our approach on standard classification and part segmentation benchmarks to demonstrate competitive results compared to the prior work. Code is publicly available at: https://github.com/engelnico/point-transformer