DualSDF: Semantic Shape Manipulation using a Two-Level Representation

TL;DR

DualSDF introduces a two-level shape representation combining detailed and abstracted shapes, enabling intuitive and semantically meaningful shape manipulation with minimal user input.

Contribution

It presents a novel dual-level shape representation with a shared latent space, facilitating high-fidelity and interpretable shape editing in 3D modeling.

Findings

Enables real-time manipulation of detailed shapes via coarse proxies.

Guides shape editing towards semantically meaningful results.

Supports complex shape modifications with minimal user input.

Abstract

We are seeing a Cambrian explosion of 3D shape representations for use in machine learning. Some representations seek high expressive power in capturing high-resolution detail. Other approaches seek to represent shapes as compositions of simple parts, which are intuitive for people to understand and easy to edit and manipulate. However, it is difficult to achieve both fidelity and interpretability in the same representation. We propose DualSDF, a representation expressing shapes at two levels of granularity, one capturing fine details and the other representing an abstracted proxy shape using simple and semantically consistent shape primitives. To achieve a tight coupling between the two representations, we use a variational objective over a shared latent space. Our two-level model gives rise to a new shape manipulation technique in which a user can interactively manipulate the coarse proxy shape and see the changes instantly mirrored in the high-resolution shape. Moreover, our model actively augments and guides the manipulation towards producing semantically meaningful shapes, making complex manipulations possible with minimal user input.