Neural Shape Parsers for Constructive Solid Geometry

TL;DR

This paper introduces CSGNe, a deep learning model that converts 2D or 3D shapes into constructive solid geometry programs, offering a fast, interpretable, and effective shape modeling approach.

Contribution

The paper presents a novel neural network architecture for shape parsing into CSG programs, including a memory-augmented encoder that improves accuracy and efficiency.

Findings

Memory-augmented architecture enhances shape reconstruction quality.

CSGNe outperforms state-of-the-art object detectors in primitive detection.

The model can be trained without explicit program annotations using policy gradients.

Abstract

Constructive Solid Geometry (CSG) is a geometric modeling technique that defines complex shapes by recursively applying boolean operations on primitives such as spheres and cylinders. We present CSGNe, a deep network architecture that takes as input a 2D or 3D shape and outputs a CSG program that models it. Parsing shapes into CSG programs is desirable as it yields a compact and interpretable generative model. However, the task is challenging since the space of primitives and their combinations can be prohibitively large. CSGNe uses a convolutional encoder and recurrent decoder based on deep networks to map shapes to modeling instructions in a feed-forward manner and is significantly faster than bottom-up approaches. We investigate two architectures for this task --- a vanilla encoder (CNN) - decoder (RNN) and another architecture that augments the encoder with an explicit memory module based on the program execution stack. The stack augmentation improves the reconstruction quality of the generated shape and learning efficiency. Our approach is also more effective as a shape primitive detector compared to a state-of-the-art object detector. Finally, we demonstrate CSGNet can be trained on novel datasets without program annotations through policy gradient techniques.