Notes on optimal approximations for importance sampling

TL;DR

This paper derives optimal conditions for function approximations in importance sampling to minimize variance, focusing on projections onto piecewise constant functions and mixture weights for target distributions, revealing differences from standard methods.

Contribution

It introduces new optimality conditions for importance sampling approximations, highlighting differences from traditional  projections and providing insights into variance minimization.

Findings

Optimal projections differ from  projections.

Derived conditions for minimizing importance sampling variance.

Analysis of mixture weights for target distribution approximation.

Abstract

In this manuscript, we derive optimal conditions for building function approximations that minimize variance when used as importance sampling estimators for Monte Carlo integration problems. Particularly, we study the problem of finding the optimal projection $g$ of an integrand $f$ onto certain classes of piecewise constant functions, in order to minimize the variance of the unbiased importance sampling estimator $E_g[f/g]$, as well as the related problem of finding optimal mixture weights to approximate and importance sample a target mixture distribution $f = \sum_i \alpha_i f_i$ with components $f_i$ in a family $\mathcal{F}$, through a corresponding mixture of importance sampling densities $g_i$ that are only approximately proportional to $f_i$. We further show that in both cases the optimal projection is different from the commonly used $\ell_1$ projection, and provide an intuitive explanation for the difference.