A Theory of Topological Derivatives for Inverse Rendering of Geometry

TL;DR

This paper develops a theoretical framework for differentiable surface evolution using topological derivatives, enabling topology changes in inverse rendering tasks for improved shape reconstruction and image vectorization.

Contribution

It introduces a novel theory for topological derivatives that facilitate topology changes during inverse rendering, surpassing sparse silhouette gradient methods.

Findings

Validated on 2D and 3D shape optimization tasks.

Enabled applications like image vectorization and shape reconstruction.

Provided insights into limitations of existing topology change methods.

Abstract

We introduce a theoretical framework for differentiable surface evolution that allows discrete topology changes through the use of topological derivatives for variational optimization of image functionals. While prior methods for inverse rendering of geometry rely on silhouette gradients for topology changes, such signals are sparse. In contrast, our theory derives topological derivatives that relate the introduction of vanishing holes and phases to changes in image intensity. As a result, we enable differentiable shape perturbations in the form of hole or phase nucleation. We validate the proposed theory with optimization of closed curves in 2D and surfaces in 3D to lend insights into limitations of current methods and enable improved applications such as image vectorization, vector-graphics generation from text prompts, single-image reconstruction of shape ambigrams and multi-view 3D reconstruction.