Node Graph Optimization Using Differentiable Proxies

TL;DR

This paper introduces a fully differentiable framework utilizing Differentiable Proxies for end-to-end optimization of material graphs, enabling matching of complex target materials even with non-differentiable components.

Contribution

It presents a novel differentiable proxy-based approach for optimizing material graphs, extending capabilities to handle non-differentiable functions in the optimization process.

Findings

Effective matching of target material structure and appearance.

Outperforms previous methods limited to differentiable functions.

Enables end-to-end gradient-based optimization of complex material graphs.

Abstract

Graph-based procedural materials are ubiquitous in content production industries. Procedural models allow the creation of photorealistic materials with parametric control for flexible editing of appearance. However, designing a specific material is a time-consuming process in terms of building a model and fine-tuning parameters. Previous work [Hu et al. 2022; Shi et al. 2020] introduced material graph optimization frameworks for matching target material samples. However, these previous methods were limited to optimizing differentiable functions in the graphs. In this paper, we propose a fully differentiable framework which enables end-to-end gradient based optimization of material graphs, even if some functions of the graph are non-differentiable. We leverage the Differentiable Proxy, a differentiable approximator of a non-differentiable black-box function. We use our framework to match structure and appearance of an output material to a target material, through a multi-stage differentiable optimization. Differentiable Proxies offer a more general optimization solution to material appearance matching than previous work.