PointInverter: Point Cloud Reconstruction and Editing via a Generative Model with Shape Priors

TL;DR

PointInverter introduces a novel method for mapping 3D point clouds into a generative model's latent space, enabling improved reconstruction and editing by resolving point correspondence issues with a shape prior-based encoder.

Contribution

It presents a new encoder for 3D point clouds that effectively inverts a sphere-guided GAN, outperforming previous methods in accuracy and correspondence resolution.

Findings

Achieves state-of-the-art inversion results for 3D point clouds.

Resolves point ordering issues during inversion.

Provides a practical tool for 3D point cloud editing.

Abstract

In this paper, we propose a new method for mapping a 3D point cloud to the latent space of a 3D generative adversarial network. Our generative model for 3D point clouds is based on SP-GAN, a state-of-the-art sphere-guided 3D point cloud generator. We derive an efficient way to encode an input 3D point cloud to the latent space of the SP-GAN. Our point cloud encoder can resolve the point ordering issue during inversion, and thus can determine the correspondences between points in the generated 3D point cloud and those in the canonical sphere used by the generator. We show that our method outperforms previous GAN inversion methods for 3D point clouds, achieving state-of-the-art results both quantitatively and qualitatively. Our code is available at https://github.com/hkust-vgd/point_inverter.