Pareto Smoothed Importance Sampling

TL;DR

This paper introduces Pareto Smoothed Importance Sampling, a novel method that stabilizes importance weights using Pareto distribution fitting, improving the accuracy and reliability of Monte Carlo estimates especially with heavy-tailed importance ratios.

Contribution

It proposes a new Pareto smoothing technique for importance sampling weights, enhancing stability and providing diagnostic tools for Monte Carlo estimators.

Findings

Empirically outperforms existing importance sampling stabilization methods.

Includes effective sample size and error estimation improvements.

Provides a convergence diagnostic based on Pareto tail fitting.

Abstract

Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right tail. This routinely occurs when there are aspects of the target distribution that are not well captured by the approximating distribution, in which case more stable estimates can be obtained by modifying extreme importance ratios. We present a new method for stabilizing importance weights using a generalized Pareto distribution fit to the upper tail of the distribution of the simulated importance ratios. The method, which empirically performs better than existing methods for stabilizing importance sampling estimates, includes stabilized effective sample size estimates, Monte Carlo error estimates, and convergence diagnostics. The presented Pareto $\hat{k}$ finite sample convergence rate diagnostic is useful for any Monte Carlo estimator.