Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes

TL;DR

The paper introduces the Adjoint Rigid Transform (ART) Network, a self-supervised neural module that aligns 3D shapes to a canonical orientation, improving performance across various 3D shape processing tasks.

Contribution

The ART Network is a novel, task-conditioned alignment module that learns canonical orientations for 3D shapes using self-supervision and rotation equivariance, applicable to both rigid and nonrigid data.

Findings

ART significantly improves task performance in shape reconstruction, interpolation, and registration.

It achieves alignment with only self-supervision, reducing manual intervention.

The method is effective for both rigid and nonrigid 3D shapes.

Abstract

Most learning methods for 3D data (point clouds, meshes) suffer significant performance drops when the data is not carefully aligned to a canonical orientation. Aligning real world 3D data collected from different sources is non-trivial and requires manual intervention. In this paper, we propose the Adjoint Rigid Transform (ART) Network, a neural module which can be integrated with a variety of 3D networks to significantly boost their performance. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks such as shape reconstruction, interpolation, non-rigid registration, and latent disentanglement. ART achieves this with self-supervision and a rotation equivariance constraint on predicted rotations. The remarkable result is that with only self-supervision, ART facilitates learning a unique canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks. We will release our code and pre-trained models for further research.