Bias-Variance Tradeoffs: Novel Applications

TL;DR

This paper explores novel applications of the bias-variance decomposition, extending from Monte Carlo integration to optimization, demonstrating that bias-variance insights can enhance Monte Carlo Optimization performance, especially in adaptive importance sampling.

Contribution

It introduces the application of bias-variance analysis to Monte Carlo Optimization, linking it to Parametric Learning and demonstrating performance improvements.

Findings

Bias-variance interpretation improves MCO performance

Application to adaptive importance sampling shows positive results

Extends bias-variance analysis beyond traditional contexts

Abstract

We present several applications of the bias-variance decomposition, beginning with straightforward Monte Carlo estimation of integrals, but progressing to the more complex problem of Monte Carlo Optimization (MCO), which involves finding a set of parameters that optimize a parameterized integral. We present the similarity of this application to that of Parametric Learning (PL). Algorithms in this field use a particular interpretation of the bias-variance trade to improve performance. This interpretation also applies to MCO, and should therefore improve performance. We verify that this is indeed the case for a particular MCO problem related to adaptive importance sampling.