HyperFlow: Representing 3D Objects as Surfaces

TL;DR

HyperFlow introduces a novel generative model that creates continuous 3D object representations as lightweight surfaces using hypernetworks and normalizing flows, improving quality and efficiency over existing methods.

Contribution

The paper proposes HyperFlow, a new approach that uses hypernetworks and continuous normalizing flows to generate surface-based 3D object representations directly from point clouds.

Findings

Produces better qualitative 3D reconstructions than competing methods.

Reduces training time by over an order of magnitude.

Effectively models 3D point density with a new spherical log-normal function.

Abstract

In this work, we present HyperFlow - a novel generative model that leverages hypernetworks to create continuous 3D object representations in a form of lightweight surfaces (meshes), directly out of point clouds. Efficient object representations are essential for many computer vision applications, including robotic manipulation and autonomous driving. However, creating those representations is often cumbersome, because it requires processing unordered sets of point clouds. Therefore, it is either computationally expensive, due to additional optimization constraints such as permutation invariance, or leads to quantization losses introduced by binning point clouds into discrete voxels. Inspired by mesh-based representations of objects used in computer graphics, we postulate a fundamentally different approach and represent 3D objects as a family of surfaces. To that end, we devise a generative model that uses a hypernetwork to return the weights of a Continuous Normalizing Flows (CNF) target network. The goal of this target network is to map points from a probability distribution into a 3D mesh. To avoid numerical instability of the CNF on compact support distributions, we propose a new Spherical Log-Normal function which models density of 3D points around object surfaces mimicking noise introduced by 3D capturing devices. As a result, we obtain continuous mesh-based object representations that yield better qualitative results than competing approaches, while reducing training time by over an order of magnitude.