Generalized active learning and design of statistical experiments for manifold-valued data

TL;DR

This paper develops a mathematical framework for efficient sampling and measurement strategies of high-dimensional, non-linear BRDF data manifolds, enhancing the process of characterizing real-world surface appearances.

Contribution

It introduces a novel theoretical foundation combining statistical design of experiments and proactive learning for manifold-valued data, specifically applied to BRDF measurements.

Findings

Framework enables more efficient sampling of BRDF manifolds

Improves accuracy of surface appearance characterization

Reduces measurement effort in complex problems

Abstract

Characterizing the appearance of real-world surfaces is a fundamental problem in multidimensional reflectometry, computer vision and computer graphics. For many applications, appearance is sufficiently well characterized by the bidirectional reflectance distribution function (BRDF). We treat BRDF measurements as samples of points from high-dimensional non-linear non-convex manifolds. BRDF manifolds form an infinite-dimensional space, but typically the available measurements are very scarce for complicated problems such as BRDF estimation. Therefore, an efficient learning strategy is crucial when performing the measurements. In this paper, we build the foundation of a mathematical framework that allows to develop and apply new techniques within statistical design of experiments and generalized proactive learning, in order to establish more efficient sampling and measurement strategies for BRDF data manifolds.

Figures

Equations

$$f_r({\omega }_i,{\omega }_r)=\frac{d{L}_r({\omega }_r)}{d{E}_i({\omega }_i)}=\frac{d{L}_r({\omega }_r)}{L_i({\omega }_i)\cos {\theta }_{i}d{\omega }_i}.$$

$$\Omega_{inc}\,=\,\bigr{\{}\omega^{(p)}_{i}\bigr{\}}^{P_{inc}}_{p=1}\,=\,\bigr{\{}\bigr{(}\theta^{(p)}_{i},\varphi^{(p)}_{i}\bigr{)}\bigr{\}}^{P_{inc}}_{p=1}\,.$$

$${\Omega }_{refl}=\bigcup _{p=1}^{P_{inc}}{\Omega }_{refl}(p),$$

$$\Omega_{refl}(p)\,=\,\biggr{\{}\omega^{(q)}_{r}\biggr{\}}^{P_{refl}(p)}_{q=1}\,=\,\biggr{\{}\bigr{(}\theta^{(q)}_{r},\varphi^{(q)}_{r}\bigr{)}\biggr{\}}^{P_{refl}(p)}_{q=1}\,,$$

$$\Omega_{meas}(n)\,=\,\biggr{\{}\bigr{(}\theta^{(p)}_{i},\varphi^{(p)}_{i},\theta^{(q)}_{r},\varphi^{(q)}_{r}\bigr{)}\,\bigr{|}\,\bigr{(}\theta^{(p)}_{i},\varphi^{(p)}_{i}\bigr{)}\in\Omega_{inc}\,,\,\bigr{(}\theta^{(q)}_{r},\varphi^{(q)}_{r}\bigr{)}\in\Omega_{refl}(p)\biggr{\}}\,,$$

$$n\,=\,\bigr{|}\Omega_{meas}(n)\bigr{|}\,=\,\sum_{p=1}^{P_{inc}}\bigr{|}\Omega_{refl}(p)\bigr{|}\,.$$

$$\Omega_{meas}(n)\,=\,\arg\min_{\Omega\,:\,Cost(\Omega)<C_{max}}\,Dist\bigr{(}\,\mathcal{E}_{n}(f;\Omega),f\,\bigr{)}$$

$$\limsup_{n\rightarrow\infty}\frac{\,Dist\bigr{(}\mathcal{E}_{n}(f;\Omega^{1}(n)),f\bigr{)}\,}{\,Dist\bigr{(}\mathcal{E}_{n}(f;\Omega^{2}(n)),f\bigr{)}\,}\,<1\,.$$

$$\limsup_{n\rightarrow\infty}\frac{\,\mathbb{E}_{\mathcal{F}^{{}^{\prime}}_{4}}\,\bigr{(}\,Dist\bigr{(}\mathcal{E}_{n}(f;\Omega^{1}(n)),f\bigr{)}\,\bigr{)}\,}{\,\mathbb{E}_{\mathcal{F}^{{}^{\prime}}_{4}}\,\bigr{(}\,Dist\bigr{(}\mathcal{E}_{n}(f;\Omega^{2}(n)),f\bigr{)}\,\bigr{)}\,}\,<1\,.$$

Full text

Generalized active learning and design of statistical experiments for manifold-valued data

Langovoy, Mikhail

*EPFL, Station 14, CH-1015 Lausanne, Switzerland

E-mail: mikhail@langovoy.com*

ABSTRACT

Characterizing the appearance of real-world surfaces is a fundamental problem in multidimensional reflectometry, computer vision and computer graphics. For many applications, appearance is sufficiently well characterized by the bidirectional reflectance distribution function (BRDF). We treat BRDF measurements as samples of points from high-dimensional non-linear non-convex manifolds. BRDF manifolds form an infinite-dimensional space, but typically the available measurements are very scarce for complicated problems such as BRDF estimation. Therefore, an efficient learning strategy is crucial when performing the measurements.

In this paper, we build the foundation of a mathematical framework that allows to develop and apply new techniques within statistical design of experiments and generalized proactive learning, in order to establish more efficient sampling and measurement strategies for BRDF data manifolds.

Keywords: Manifold-valued data, BRDF, proactive learning, sampling strategy.

1 Introduction

In computer graphics and computer vision, usually either physically inspired analytic reflectance models, like Cook and Torrance (1981) or He et al. (1991), or parametric reflectance models chosen via qualitative criteria, like Phong (1975), or Lafortune et al. (1997), are used to model BRDFs. These BRDF models are only crude approximations of the reflectance of real materials. In multidimensional reflectometry, an alternative approach is usually taken. One directly measures values of the BRDF for different combinations of the incoming and outgoing angles and then fits the measured data to a selected analytic model using optimization techniques.

There were numerous efforts to use modern machine learning techniques to construct data-driven BRDF models. Brady et al. (2014) proposed a method to generate new analytical BRDFs using a heuristic distance-based search procedure called Genetic Programming. In Brochu et al. (2008), an active learning algorithm using discrete perceptional data was developed and applied to learning parameters of BRDF models such as the Ashikhmin - Shirley model Ashikhmin and Shirley (2000), while Langovoy et al. (2016) treated active learning for the Cook - Torrance model Cook and Torrance (1981). Analysis of BRDF data with statistical and machine learning methods was discussed in Langovoy (2015b), Langovoy (2015a), Sole et al. (2018), Doctor and Byers (2018).

2 Active learning and design of experiments

In general, BRDF is a 5-dimensional manifold, having 4 angular and 1 wavelength dimension. Note that even a set of 1-dimensional manifolds is infinite-dimensional (and $k$-dimensional manifolds are not to be confused with parametric $k$-dimensional families of functions). At the same time, a typical measuring device only takes between 50 and 1000 points for all the BRDF layers together. In view of this, the available measurement points are indeed very scarce for a complicated problem such as BRDF estimation. Therefore, an efficient sampling strategy is required when performing the measurements. Since sets of BRDF measurements are, in fact, observed random manifolds, we are dealing here with manifold-valued data.

Statistical design of experiments (see Fisher et al. (1960), Cox and Reid (2000)) is a well-developed area of quantitative data analysis. However, previous research in this field was often more concerned with (important) topics such as manipulation checks, interactions between factors, delayed effects, repeatability, among many others. This shifted the focus away from considering design of statistical experiments on structured, constrained, or infinite-dimensional data. In contrast,BRDF measurements are carried out in strictly defined settings and by qualified experts. Therefore, there is less room for human or random errors and influences. On the other hand, BRDF measurements are collections of points representing manifolds, so defining even the simplest statistical quantities in this case turns out to be a nontrivial and conceptual task.

Overall, our methodology represents a far-reaching generalization of the active machine learning framework, also generalizing the proactive learning setup of Donmez and Carbonell (2008). Active learning, as a special case of semi-supervised machine learning, oftentimes deals with finite sets of labels and aims at solving classification or clustering problems with a finite number of classes. While there have been a number of promising practical applications, most of the existing theory deals with analysis of performance of specific algorithms (query by committee, $A^{2}$ algorithm, or importance-weighted approach, among a few others) under rather restrictive conditions on the loss functions, incoming distributions, and other components of the learning model. For recent developments, we refer to Agarwal et al. (2013), Beygelzimer et al. (2009), Dasgupta and Hsu (2008).

3 Main definition

In the most basic case, the bidirectional reflectance distribution function (BRDF), $f_{r}(\omega_{i},\,\omega_{r}))$ is a four-dimensional function that defines how light is reflected at an opaque surface. The function takes a negative incoming light direction, $\omega_{i}$, and outgoing direction, $\omega_{r}$, both defined with respect to the surface normal $\mathbf{n}$, and returns the ratio of reflected radiance exiting along $\omega_{r}$ to the irradiance incident on the surface from direction $\omega_{i}$. The BRDF was first defined by Nicodemus in Nicodemus (1965). The defining equation is:

[TABLE]

where $L$ is radiance, or power per unit solid-angle-in-the-direction-of-a-ray per unit projected-area-perpendicular-to-the-ray, $E$ is irradiance, or power per unit surface area, and $\theta_{i}$ is the angle between $\omega_{i}$ and the surface normal, $\mathbf{n}$. The index $i$ indicates incident light, whereas the index $r$ indicates reflected light.

Suppose we have measurements of a BRDF available for the set of incoming angles

[TABLE]

Here $P_{inc}\geq 1$ is the total number of incoming angles where the measurements were taken. Say that for an incoming angle \bigr{\{}\omega^{(p)}_{i}\bigr{\}} we have measurements available for angles from the set of reflection angles

[TABLE]

where

[TABLE]

where \bigr{\{}P_{refl}(p)\bigr{\}}^{P_{inc}}_{p=1} are (possibly different) numbers of measurements taken for corresponding incoming angles. Our aim is to infer the BRDF manifold (1) from the above observations.

In general, the connection between the true BRDF and its measurements is described via a stochastic transformation $T$, i.e. f(\omega_{i},\,\omega_{r})\,=\,T\bigr{(}f_{r}(\omega_{i},\,\omega_{r})\bigr{)}, where $T\,:\,\mathcal{M}\times\mathcal{P}\times\mathcal{F}_{4}\,\rightarrow\,F_{4}$, with $\mathcal{M}\,=\,(M,\mathfrak{A},\mu)$ is an (unknown) measurable space, $\mathcal{P}\,=\,(\Pi,\mathfrak{P},\mathbb{P})$ is an unknown probability space, $\mathcal{F}_{4}$ is the space of all Helmholtz-invariant energy preserving 4-dimensional BRDFs, and $F_{4}$ is the set of all functions of 4 arguments on the 3-dimensional unit sphere $S^{3}$ in $\mathbb{R}^{4}$.

In order to evaluate the influence of measurement errors and to be able to measure the quality of fit of BRDF models, one needs a ”measure of distance” between BRDFs. There are many choices of distances and quasi-distances available: $L_{p}$, $1\leq p<+\infty$, $L_{\infty}$, Sobolev distances, Kullback-Leibler information divergence Kullback and Leibler (1951), Mahalanobis (1936), chi-squared distance used in correspondence analysis Langovaya et al. (2013). In computer science literature on BRDFs, there are few papers that study the quality of fit of BRDF models to real data. Most of these studies use the (most standard) $L_{2}$-norm. An alternative approach was taken in Langovoy et al. (2014), where a perception-inspired quasi-metric for the space of BRDFs was proposed.

4 Active manifold learning strategies

In BRDF sampling, the equispaced-angular grid pictured in Figure 1 is the standard. However, as was shown in Langovoy et al. (2016), this choice of measurement points leads to very inefficient sampling. Another strategy is in using uniformly distributed points on a sphere, see Figure 1. Since it was already understood in the community (see Höpe and Hauer (2010)) that the standard grid is suboptimal, there were multiple heuristic attempts to propose trickier grids that better reflect the typical structure of BRDF models. A good example is shown in Figure 1. Ideally, the main goal of this research is to find the best sampling strategy; this strategy has to retain its optimality at least for a class of reasonable criteria, and for a sufficiently general classes of both BRDFs as well as of estimating procedures.

On the other hand, any result showing that new strategy is better than the default strategy, at least for one specific loss function, for one specific BRDF, and one specific estimating procedure, is already instrumental in understanding the general picture of learning BRDF manifolds from scarce expensive data. This basic case is straightforwardly formulated in the language of mathematical optimization, so we are able to obtain theoretical guarantees on learning accuracy, at least for some special cases. Let us outline a possible mathematical framework for BRDF sampling, in a basic case to begin with.

Consider BRDF $f\in\mathcal{F}_{4}$. Suppose that $f$ is measured on the finite set $\Omega_{meas}(n)$

[TABLE]

where

[TABLE]

Definition 1**.**

Cost function $Cost$ of a measurement configuration $\Omega_{meas}$ is a Lebesgue measurable function Cost\,:\,\mathbb{R}^{\bigr{|}\Omega_{meas}\bigr{|}}\,\rightarrow\,\mathbb{R}_{+}.

Let $Dist$ be a function (measurable for a suitably chosen $\sigma$-algebra) such that $Dist\,:\,\mathcal{F}_{4}\times\mathcal{F}_{4}\,\rightarrow\,\mathbb{R}_{+}$. For our purposes, we typically like $Dist$ to be inducing either a quasi-distance or a pseudo-distance on $\mathcal{F}^{0}_{4}$, where $\mathcal{F}^{0}_{4}\,\subseteq\,\mathcal{F}_{4}$ is a sufficiently reach subset. As an example, a perception-based $\mu_{BRDF}$ from Langovoy et al. (2014), was often used in our practical experiments. Standard $L_{p}$-distances are easier for theoretical comparisons.

Definition 2**.**

Sampling strategy $\Omega$ is a sequence $\Omega=\{\Omega_{n_{0}}\}_{n_{0}=1}^{\infty}$ where for each $n_{0}$ there exists an integer $n\geq n_{0}$ such that $\Omega_{n_{0}}\,=\,\Omega_{meas}(n)$ for some measurement configuration $\Omega_{meas}(n)$ defined according to (4), and for any integers $n_{1}\leq n_{2}$ it holds that \bigr{|}\Omega_{n_{1}}\bigr{|}\,\leq\,\bigr{|}\Omega_{n_{2}}\bigr{|}.

Consider arbitrary fixed statistical estimator of BRDFs, $\mathcal{E}_{n}\,:\,\mathbb{R}^{n}\,\rightarrow\,\mathcal{F}_{4}$.

Definition 3**.**

Let $\Omega=\{\Omega_{n_{0}}\}_{n_{0}=1}^{\infty}$ be a sampling strategy, and suppose that $C_{max}\,:\,\mathbb{N}\,\rightarrow\,\mathbb{R}_{+}$ be a known function. We say that the strategy $\Omega$ has uniformly admissible costs with the majorant $C_{max}$, if for all $n\geq 1$ it holds that $Cost(\Omega_{n})\,<\,C_{max}(n)$. We say that $\Omega$ has asymptotically uniformly admissible costs with the majorant $C_{max}$, if there exist $n_{min}\in\mathbb{N}$ such that for all $n\geq n_{min}$ it holds that $Cost(\Omega_{n})\,<\,C_{max}(n)$.

Consider two sampling strategies: $\Omega^{1}$, $\Omega^{2}$. Suppose that both strategies have uniformly admissible costs. The problem of generalized active learning for BRDF sampling can be stated in the following way: find a sampling strategy \Omega_{meas}\,=\,\bigr{\{}\Omega_{meas}(n)\bigr{\}}_{n=n_{min}}^{\infty} such that for all $n\geq n_{min}$

[TABLE]

Definition 4**.**

Suppose $f\in\mathcal{F}_{4}$ is a particular (possibly unknown) BRDF. Let $\Omega^{1}$, $\Omega^{2}$ sampling strategies be sampling strategies with $C_{max}$-uniformly admissible costs. We say that strategy $\Omega^{1}$ is asymptotically more efficient for learning $f$ than the strategy $\Omega^{2}$, and write $\Omega^{1}\,\succcurlyeq_{f}\,\Omega^{2}$, iff

[TABLE]

Notice that, for the task of evaluating sampling quality, expected errors (over classes of BRDFs) are more interesting than maximal errors (over the same classes). Indeed, maximal errors are often dominated by degenerate counterexamples, while we are interested in a typical case behavior of our learning procedures. Therefore, we are typically interested in expected errors of the form $\mathbb{E}_{\mathcal{F}^{{}^{\prime}}_{4}}\,(\,Dist(\mathcal{E}_{n}(f;\Omega(n)),f)\,)\,\rightarrow\,\min\,$, where $\mathcal{F}^{{}^{\prime}}_{4}\,\subseteq\,\mathcal{F}_{4}$ is a sufficiently reach subset. Clearly, the choice of quasi-metric $Dist$ plays a crucial role.

Definition 5**.**

Suppose $\mathcal{F}^{{}^{\prime}}_{4}\,\subseteq\,\mathcal{F}_{4}$ is a subset of the set of BRDFs. Let $\Omega^{1}$, $\Omega^{2}$ sampling strategies be sampling strategies with $C_{max}$-uniformly admissible costs. We say that strategy $\Omega^{1}$ is asymptotically more efficient for learning BRDFs of the class $\mathcal{F}^{{}^{\prime}}_{4}$ than the strategy $\Omega^{2}$, and write $\Omega^{1}\,\succcurlyeq\,\Omega^{2}$, iff

[TABLE]

Notice that this problem is neither a classification nor a regression task, as we are picking points to estimate manifolds from noisy data.

A special case of this Definition was used in Langovoy et al. (2016) in order to propose more efficient BRDF sampling strategies for industrial applications.

5 Conclusions

BRDF manifolds form an infinite-dimensional space, but typically the available measurements are very scarce and expensive. Therefore, an efficient sampling strategy is crucial when performing the measurements. We built a mathematical framework that allows to develop and apply new techniques within statistical design of experiments and generalized proactive learning, in order to establish more efficient sampling and measurement strategies for manifold-valued BRDF data.