Error Bounds for Importance Sampling with Estimated Proposal Distributions

TL;DR

This paper develops non-asymptotic error bounds for importance sampling using data-driven proposals, particularly kernel density estimators, providing practical guidance for their use.

Contribution

It derives explicit error bounds for importance sampling with estimated proposals, including KDEs from Markov chains, and separates Monte Carlo error from proposal approximation error.

Findings

Monte Carlo error scales as n^{-1/2}

Proposal approximation error depends on KDE MIAE and MISE

Provides quantitative guarantees for KDE-based importance sampling

Abstract

Importance sampling with data-driven proposal distributions is widely used in practice. A common workflow first generates an auxiliary sample of size $N$ from an approximation of the target distribution, constructs a density estimate $\hat q$ such as a kernel density estimator (KDE), and then draws $n$ importance samples from this learned proposal. Despite its practical relevance, the theoretical properties of this hierarchical procedure remain poorly understood, since classical importance sampling theory assumes a fixed proposal. We address this gap by deriving non-asymptotic error bounds for standard, defensive, and self-normalized importance sampling estimators with random proposals. Our results separate the Monte Carlo error, scaling as $n^{-1/2}$, from the proposal approximation error measured through the mean integrated absolute and squared errors (MIAE and MISE) of $\hat q$. To obtain explicit convergence rates in $(N,n)$, we establish MIAE and MISE bounds for KDEs constructed from geometrically ergodic Markov chains in stationary and non-stationary regimes. Combining these results yields quantitative guarantees for importance sampling with KDE-based proposals. Our theory provides practical guidance for selecting defensive mixture weights in a nonparametric importance sampling framework.