Joint Sampling and Optimisation for Inverse Rendering

TL;DR

This paper introduces a novel method combining sampling and optimization that reduces gradient variance in inverse rendering, leading to faster convergence in complex inverse problems.

Contribution

It develops a theoretical framework for interleaving sampling and optimization using low-variance finite-difference estimators and integrates this with Adam for improved inverse rendering.

Findings

Speeds up convergence in inverse path tracing tasks

Reduces gradient variance effectively during optimization

Demonstrates improved efficiency over traditional methods

Abstract

When dealing with difficult inverse problems such as inverse rendering, using Monte Carlo estimated gradients to optimise parameters can slow down convergence due to variance. Averaging many gradient samples in each iteration reduces this variance trivially. However, for problems that require thousands of optimisation iterations, the computational cost of this approach rises quickly. We derive a theoretical framework for interleaving sampling and optimisation. We update and reuse past samples with low-variance finite-difference estimators that describe the change in the estimated gradients between each iteration. By combining proportional and finite-difference samples, we continuously reduce the variance of our novel gradient meta-estimators throughout the optimisation process. We investigate how our estimator interlinks with Adam and derive a stable combination. We implement our method for inverse path tracing and demonstrate how our estimator speeds up convergence on difficult optimisation tasks.