Stochastic Gradient Estimation for Higher-order Differentiable Rendering

TL;DR

This paper introduces methods to compute higher-order derivatives of rendering operators, enhancing optimization techniques in inverse rendering tasks through importance sampling and aggregate sampling strategies.

Contribution

It presents a novel approach for higher-order differential computation in rendering, applicable to rasterization and path tracing, improving convergence in inverse rendering optimization.

Findings

Higher-order derivatives improve optimizer convergence

Importance sampling of rendering differentials is effective

Aggregate sampling enhances differential estimation

Abstract

We derive methods to compute higher order differentials (Hessians and Hessian-vector products) of the rendering operator. Our approach is based on importance sampling of a convolution that represents the differentials of rendering parameters and shows to be applicable to both rasterization and path tracing. We further suggest an aggregate sampling strategy to importance-sample multiple dimensions of one convolution kernel simultaneously. We demonstrate that this information improves convergence when used in higher-order optimizers such as Newton or Conjugate Gradient relative to a gradient descent baseline in several inverse rendering tasks.