Transformers in 3D Point Clouds: A Survey

TL;DR

This survey comprehensively reviews the application of Transformer architectures in 3D point cloud analysis, covering theory, taxonomy, variants, and performance comparisons across tasks, highlighting current challenges and future directions.

Contribution

It provides the first systematic survey of Transformer methods in 3D point cloud processing, including taxonomy, analysis of variants, and performance evaluation across multiple tasks.

Findings

Transformers show superior performance in 3D point cloud tasks.

Variants of self-attention improve efficiency and effectiveness.

Transformers outperform traditional methods in classification, segmentation, and detection.

Abstract

Transformers have been at the heart of the Natural Language Processing (NLP) and Computer Vision (CV) revolutions. The significant success in NLP and CV inspired exploring the use of Transformers in point cloud processing. However, how do Transformers cope with the irregularity and unordered nature of point clouds? How suitable are Transformers for different 3D representations (e.g., point- or voxel-based)? How competent are Transformers for various 3D processing tasks? As of now, there is still no systematic survey of the research on these issues. For the first time, we provided a comprehensive overview of increasingly popular Transformers for 3D point cloud analysis. We start by introducing the theory of the Transformer architecture and reviewing its applications in 2D/3D fields. Then, we present three different taxonomies (i.e., implementation-, data representation-, and task-based), which can classify current Transformer-based methods from multiple perspectives. Furthermore, we present the results of an investigation of the variants and improvements of the self-attention mechanism in 3D. To demonstrate the superiority of Transformers in point cloud analysis, we present comprehensive comparisons of various Transformer-based methods for classification, segmentation, and object detection. Finally, we suggest three potential research directions, providing benefit references for the development of 3D Transformers.