PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

TL;DR

PoinTr introduces a transformer-based approach for point cloud completion that models local geometric relationships explicitly, leading to significant improvements over existing methods on new and existing benchmarks.

Contribution

The paper proposes a novel transformer encoder-decoder architecture with geometry-aware blocks for more effective point cloud completion from partial data.

Findings

Outperforms state-of-the-art methods on multiple benchmarks

Introduces new challenging benchmarks for real-world scenarios

Effectively preserves detailed geometric structures during completion

Abstract

Point clouds captured in real-world applications are often incomplete due to the limited sensor resolution, single viewpoint, and occlusion. Therefore, recovering the complete point clouds from partial ones becomes an indispensable task in many practical applications. In this paper, we present a new method that reformulates point cloud completion as a set-to-set translation problem and design a new model, called PoinTr that adopts a transformer encoder-decoder architecture for point cloud completion. By representing the point cloud as a set of unordered groups of points with position embeddings, we convert the point cloud to a sequence of point proxies and employ the transformers for point cloud generation. To facilitate transformers to better leverage the inductive bias about 3D geometric structures of point clouds, we further devise a geometry-aware block that models the local geometric relationships explicitly. The migration of transformers enables our model to better learn structural knowledge and preserve detailed information for point cloud completion. Furthermore, we propose two more challenging benchmarks with more diverse incomplete point clouds that can better reflect the real-world scenarios to promote future research. Experimental results show that our method outperforms state-of-the-art methods by a large margin on both the new benchmarks and the existing ones. Code is available at https://github.com/yuxumin/PoinTr