Stochastic adaptation of importance sampler

TL;DR

This paper introduces a stochastic approximation-based iterative method to adaptively optimize the proposal distribution in importance sampling, enhancing efficiency and convergence towards the target distribution.

Contribution

It presents a novel adaptive importance sampling technique using stochastic approximation, compatible with general iterative optimization algorithms like minorization-maximization.

Findings

Effective in reducing Kullback divergence

Demonstrated on simple examples

Improves importance sampling efficiency

Abstract

Improving efficiency of importance sampler is at the center of research in Monte Carlo methods. While adaptive approach is usually difficult within the Markov Chain Monte Carlo framework, the counterpart in importance sampling can be justified and validated easily. We propose an iterative adaptation method for learning the proposal distribution of an importance sampler based on stochastic approximation. The stochastic approximation method can recruit general iterative optimization techniques like the minorization-maximization algorithm. The effectiveness of the approach in optimizing the Kullback divergence between the proposal distribution and the target is demonstrated using several simple examples.