Learned Interpolation for 3D Generation

TL;DR

This paper introduces a novel learned interpolation method for 3D point clouds that produces realistic, creative, and sculpture-ready 3D shapes by encoding prior knowledge about real-world objects.

Contribution

It proposes a new approach to interpolate in learned latent spaces for 3D shapes, enabling both realistic and creative shape generation.

Findings

Generated point clouds are both realistic and novel.

Method can produce sculpture-suitable 3D models.

Supports creative and unexpected shape synthesis.

Abstract

In order to generate novel 3D shapes with machine learning, one must allow for interpolation. The typical approach for incorporating this creative process is to interpolate in a learned latent space so as to avoid the problem of generating unrealistic instances by exploiting the model's learned structure. The process of the interpolation is supposed to form a semantically smooth morphing. While this approach is sound for synthesizing realistic media such as lifelike portraits or new designs for everyday objects, it subjectively fails to directly model the unexpected, unrealistic, or creative. In this work, we present a method for learning how to interpolate point clouds. By encoding prior knowledge about real-world objects, the intermediate forms are both realistic and unlike any existing forms. We show not only how this method can be used to generate "creative" point clouds, but how the method can also be leveraged to generate 3D models suitable for sculpture.