Heretical Multiple Importance Sampling

TL;DR

This paper introduces a novel heretical MIS framework that performs a posteriori clustering to reduce variance in importance sampling, achieving lower MSE with less computational effort compared to existing methods.

Contribution

It proposes a new heretical MIS approach that adaptively clusters proposals after sampling to minimize variance, offering a bias-variance trade-off and improved efficiency.

Findings

Heretical MIS reduces variance significantly.

It outperforms standard and partial MIS in MSE.

Achieves near-DM performance with less computation.

Abstract

Multiple Importance Sampling (MIS) methods approximate moments of complicated distributions by drawing samples from a set of proposal distributions. Several ways to compute the importance weights assigned to each sample have been recently proposed, with the so-called deterministic mixture (DM) weights providing the best performance in terms of variance, at the expense of an increase in the computational cost. A recent work has shown that it is possible to achieve a trade-off between variance reduction and computational effort by performing an a priori random clustering of the proposals (partial DM algorithm). In this paper, we propose a novel "heretical" MIS framework, where the clustering is performed a posteriori with the goal of reducing the variance of the importance sampling weights. This approach yields biased estimators with a potentially large reduction in variance. Numerical examples show that heretical MIS estimators can outperform, in terms of mean squared error (MSE), both the standard and the partial MIS estimators, achieving a performance close to that of DM with less computational cost.