Interpretable Disentangled Parametrization of Measured BRDF with $\beta$-VAE

TL;DR

This paper introduces an interpretable, disentangled parametrization of measured BRDFs using a $eta$-VAE, enabling artistic control, expansion with new parameters, and real-time rendering without requiring test subject investigations.

Contribution

The work presents a novel $eta$-VAE-based approach for interpretable BRDF parametrization that allows for expansion, flexible editing, and real-time rendering, improving over existing methods.

Findings

Disentangled parameters correspond to perceptually meaningful factors.

The method enables expansion with new parameters post-training.

Real-time rendering and interpolation are achieved with the new parametrization.

Abstract

Finding a low dimensional parametric representation of measured BRDF remains challenging. Currently available solutions are either not interpretable, or rely on limited analytical solutions, or require expensive test subject based investigations. In this work, we strive to establish a parametrization space that affords the data-driven representation variance of measured BRDF models while still offering the artistic control of parametric analytical BRDFs. We present a machine learning approach that generates an interpretable disentangled parameter space. A disentangled representation is one in which each parameter is responsible for a unique generative factor and is insensitive to the ones encoded by the other parameters. To that end, we resort to a $\beta$-Variational AutoEncoder ($\beta$-VAE), a specific architecture of Deep Neural Network (DNN). After training our network, we analyze the parametrization space and interpret the learned generative factors utilizing our visual perception. It should be noted that perceptual analysis is called upon downstream of the system for interpretation purposes compared to most other existing methods where it is used upfront to elaborate the parametrization. In addition to that, we do not need a test subject investigation. A novel feature of our interpretable disentangled parametrization is the post-processing capability to incorporate new parameters along with the learned ones, thus expanding the richness of producible appearances. Furthermore, our solution allows more flexible and controllable material editing possibilities than manifold exploration. Finally, we provide a rendering interface, for real-time material editing and interpolation based on the presented new parametrization system.