BR-SNIS: Bias Reduced Self-Normalized Importance Sampling

TL;DR

BR-SNIS is a novel importance sampling method that significantly reduces bias while maintaining low variance, using iterated sampling-importance resampling with the same computational complexity as standard SNIS.

Contribution

The paper introduces BR-SNIS, a bias reduction technique for self-normalized importance sampling that leverages ISIR, with theoretical guarantees and practical effectiveness.

Findings

Reduces bias significantly compared to SNIS.

Maintains similar computational complexity as SNIS.

Provides rigorous theoretical bounds and numerical validation.

Abstract

Importance Sampling (IS) is a method for approximating expectations under a target distribution using independent samples from a proposal distribution and the associated importance weights. In many applications, the target distribution is known only up to a normalization constant, in which case self-normalized IS (SNIS) can be used. While the use of self-normalization can have a positive effect on the dispersion of the estimator, it introduces bias. In this work, we propose a new method, BR-SNIS, whose complexity is essentially the same as that of SNIS and which significantly reduces bias without increasing the variance. This method is a wrapper in the sense that it uses the same proposal samples and importance weights as SNIS, but makes clever use of iterated sampling--importance resampling (ISIR) to form a bias-reduced version of the estimator. We furnish the proposed algorithm with rigorous theoretical results, including new bias, variance and high-probability bounds, and these are illustrated by numerical examples.