Iterative Transformer Network for 3D Point Cloud

TL;DR

This paper introduces IT-Net, an iterative transformer network that canonicalizes partial 3D point clouds to improve shape understanding tasks like classification and segmentation, especially in unaligned, real-world data.

Contribution

The paper presents a novel iterative transformer module that estimates and canonicalizes object pose from partial point clouds, enhancing 3D shape analysis.

Findings

IT-Net effectively estimates object pose from partial point clouds.

IT-Net outperforms existing transformer networks on shape classification and segmentation.

IT-Net operates without needing complete object models.

Abstract

3D point cloud is an efficient and flexible representation of 3D structures. Recently, neural networks operating on point clouds have shown superior performance on 3D understanding tasks such as shape classification and part segmentation. However, performance on such tasks is evaluated on complete shapes aligned in a canonical frame, while real world 3D data are partial and unaligned. A key challenge in learning from partial, unaligned point cloud data is to learn features that are invariant or equivariant with respect to geometric transformations. To address this challenge, we propose the Iterative Transformer Network (IT-Net), a network module that canonicalizes the pose of a partial object with a series of 3D rigid transformations predicted in an iterative fashion. We demonstrate the efficacy of IT-Net as an anytime pose estimator from partial point clouds without using complete object models. Further, we show that IT-Net achieves superior performance over alternative 3D transformer networks on various tasks, such as partial shape classification and object part segmentation.