MeshSDF: Differentiable Iso-Surface Extraction

TL;DR

MeshSDF introduces a novel differentiable method to extract explicit mesh surfaces from deep implicit fields, enabling end-to-end learning and optimization for 3D shape tasks.

Contribution

The paper presents MeshSDF, a differentiable approach to generate explicit surface meshes from deep signed distance functions, allowing topology variation and improved 3D modeling.

Findings

Outperforms state-of-the-art in single-view reconstruction

Enables end-to-end shape optimization with differentiable meshes

Supports topology changes in mesh representations

Abstract

Geometric Deep Learning has recently made striking progress with the advent of continuous Deep Implicit Fields. They allow for detailed modeling of watertight surfaces of arbitrary topology while not relying on a 3D Euclidean grid, resulting in a learnable parameterization that is not limited in resolution. Unfortunately, these methods are often not suitable for applications that require an explicit mesh-based surface representation because converting an implicit field to such a representation relies on the Marching Cubes algorithm, which cannot be differentiated with respect to the underlying implicit field. In this work, we remove this limitation and introduce a differentiable way to produce explicit surface mesh representations from Deep Signed Distance Functions. Our key insight is that by reasoning on how implicit field perturbations impact local surface geometry, one can ultimately differentiate the 3D location of surface samples with respect to the underlying deep implicit field. We exploit this to define MeshSDF, an end-to-end differentiable mesh representation which can vary its topology. We use two different applications to validate our theoretical insight: Single-View Reconstruction via Differentiable Rendering and Physically-Driven Shape Optimization. In both cases our differentiable parameterization gives us an edge over state-of-the-art algorithms.