Neural Enhancement of Analytical Appearance Models

TL;DR

This paper introduces neural enhancement, a framework that improves analytical appearance models by replacing key nodes with neural networks, resulting in more accurate, compact, and efficient models compatible with standard rendering pipelines.

Contribution

The authors propose a novel neural enhancement method that automatically identifies and replaces parts of analytical models with neural networks using a hypercube-based search.

Findings

Enhanced analytical BRDF models show improved accuracy and expressiveness.

The method produces models that are compact and computationally efficient.

Enhanced models outperform state-of-the-art fitting techniques on measured reflectance data.

Abstract

Traditional analytical reflectance models, while compact and interpretable, lack the capacity to accurately represent physical measurements. Recent neural models, which closely fit input data, are less generalizable and often more expensive to store and evaluate. To combine the strengths and overcome the limitations of these two classes of models, we present neural enhancement, a novel framework to boost an input analytical appearance model, by identifying and replacing its key computational nodes/operators with small-scale multi-layer perceptrons. This allows us to leverage the computational graph structure of the original model, while improving its expressiveness at a modest cost. To make the enhancement computationally tractable, we propose a hypercube-based search to automatically and efficiently identify the node(s) and/or operator(s) to be replaced towards maximal gain in a differentiable fashion. We enhance a number of common analytical BRDF models. The results are, at once accurate, compact and efficient, and compare favorably with state-of-the-art work on fitting measured reflectance and bidirectional texture functions. Finally, our models are fully compatible with any standard rasterization or ray-tracing pipeline.