Optimality in importance sampling: a gentle survey

TL;DR

This paper provides a comprehensive review of optimality concepts in importance sampling, covering theoretical frameworks, adaptive methods, and applications like ABC and reinforcement learning, with comparisons and insights.

Contribution

It offers an exhaustive survey of optimality in importance sampling, integrating various frameworks, theoretical analyses, and empirical evaluations across multiple applications.

Findings

Multiple proposal densities improve sampling efficiency.

Optimality frameworks guide adaptive importance sampling.

Empirical comparisons highlight strengths and limitations.

Abstract

The performance of the Monte Carlo sampling methods relies on the crucial choice of a proposal density. The notion of optimality is fundamental to design suitable adaptive procedures of the proposal density within Monte Carlo schemes. This work is an exhaustive review around the concept of optimality in importance sampling. Several frameworks are described and analyzed, such as the marginal likelihood approximation for model selection, the use of multiple proposal densities, a sequence of tempered posteriors, and noisy scenarios including the applications to approximate Bayesian computation (ABC) and reinforcement learning, to name a few. Some theoretical and empirical comparisons are also provided.