PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows

TL;DR

PointFlow introduces a probabilistic framework using continuous normalizing flows to generate and reconstruct high-quality 3D point clouds, enabling flexible sampling and improved performance over previous methods.

Contribution

We develop a hierarchical probabilistic model with continuous normalizing flows for 3D point cloud generation and reconstruction, providing a novel approach to shape and point sampling.

Findings

Achieves state-of-the-art results in point cloud generation

Faithfully reconstructs point clouds with high fidelity

Learns useful representations in an unsupervised manner

Abstract

As 3D point clouds become the representation of choice for multiple vision and graphics applications, the ability to synthesize or reconstruct high-resolution, high-fidelity point clouds becomes crucial. Despite the recent success of deep learning models in discriminative tasks of point clouds, generating point clouds remains challenging. This paper proposes a principled probabilistic framework to generate 3D point clouds by modeling them as a distribution of distributions. Specifically, we learn a two-level hierarchy of distributions where the first level is the distribution of shapes and the second level is the distribution of points given a shape. This formulation allows us to both sample shapes and sample an arbitrary number of points from a shape. Our generative model, named PointFlow, learns each level of the distribution with a continuous normalizing flow. The invertibility of normalizing flows enables the computation of the likelihood during training and allows us to train our model in the variational inference framework. Empirically, we demonstrate that PointFlow achieves state-of-the-art performance in point cloud generation. We additionally show that our model can faithfully reconstruct point clouds and learn useful representations in an unsupervised manner. The code will be available at https://github.com/stevenygd/PointFlow.