Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs

TL;DR

This paper introduces an octree-based deep convolutional decoder architecture that efficiently generates high-resolution 3D shapes, significantly reducing memory and computation compared to traditional voxel-based methods.

Contribution

It proposes a novel octree-structured neural network that predicts octree structure and occupancy, enabling high-resolution 3D output with limited resources.

Findings

Achieves high-resolution 3D shape generation efficiently

Reduces memory and computational complexity compared to voxel grids

Demonstrates versatility across multiple 3D tasks

Abstract

We present a deep convolutional decoder architecture that can generate volumetric 3D outputs in a compute- and memory-efficient manner by using an octree representation. The network learns to predict both the structure of the octree, and the occupancy values of individual cells. This makes it a particularly valuable technique for generating 3D shapes. In contrast to standard decoders acting on regular voxel grids, the architecture does not have cubic complexity. This allows representing much higher resolution outputs with a limited memory budget. We demonstrate this in several application domains, including 3D convolutional autoencoders, generation of objects and whole scenes from high-level representations, and shape from a single image.