Generalizing the Balance Heuristic Estimator in Multiple Importance Sampling

TL;DR

This paper introduces a generalized family of multiple importance sampling estimators that extend the balance heuristic, providing a framework for optimal parameter selection to minimize variance and improve estimation accuracy.

Contribution

A new generalized framework for multiple importance sampling that includes the balance heuristic as a special case and offers optimal parameter choices for variance reduction.

Findings

The generalized estimator always outperforms the balance heuristic in variance, except in degenerate cases.

Theoretical analysis provides new upper bounds for the balance heuristic estimator.

Numerical examples demonstrate the variance gap between the estimators.

Abstract

In this paper, we propose a novel and generic family of multiple importance sampling estimators. We first revisit the celebrated balance heuristic estimator, a widely used Monte Carlo technique for the approximation of intractable integrals. Then, we establish a generalized framework for the combination of samples simulated from multiple proposals. We show that the novel framework contains the balance heuristic as a particular case. In addition, we study the optimal choice of the free parameters in such a way the variance of the resulting estimator is minimized. A theoretical variance study shows the optimal solution is always better than the balance heuristic estimator (except in degenerate cases where both are the same). As a side result of this analysis, we also provide new upper bounds for the balance heuristic estimator. Finally, we show the gap in the variance of both estimators by means of five numerical examples.