Robust Importance Sampling with Adaptive Winsorization

TL;DR

This paper introduces an adaptive winsorization method for importance sampling that balances bias and variance, leading to estimators with smaller mean squared error and deviation, backed by finite-sample guarantees.

Contribution

It proposes a novel winsorizing procedure based on the Balancing Principle that adaptively selects thresholds to optimize importance sampling estimators.

Findings

Smaller mean squared error compared to existing methods

Reduced mean absolute deviation in experiments

Finite-sample optimality guarantees under minimal assumptions

Abstract

Importance sampling is a widely used technique to estimate properties of a distribution. This paper investigates trading-off some bias for variance by adaptively winsorizing the importance sampling estimator. The novel winsorizing procedure, based on the Balancing Principle (or Lepskii's Method), chooses a threshold level among a pre-defined set by roughly balancing the bias and variance of the estimator when winsorized at different levels. As a consequence, it provides a principled way to perform winsorization with finite-sample optimality guarantees under minimal assumptions. In various examples, the proposed estimator is shown to have smaller mean squared error and mean absolute deviation than leading alternatives.