Adaptive Importance Sampling for Estimation in Structured Domains

TL;DR

This paper introduces adaptive importance sampling methods that iteratively improve sampling distributions for estimating complex sums and integrals, validated through experiments on influence diagrams.

Contribution

It proposes stochastic-gradient-descent algorithms for adaptively optimizing sampling distributions to minimize variance in high-dimensional estimation tasks.

Findings

The methods effectively reduce variance in sampling estimates.

Adaptive sampling improves accuracy over static approaches.

Empirical validation demonstrates practical benefits in influence diagram evaluation.

Abstract

Sampling is an important tool for estimating large, complex sums and integrals over high dimensional spaces. For instance, important sampling has been used as an alternative to exact methods for inference in belief networks. Ideally, we want to have a sampling distribution that provides optimal-variance estimators. In this paper, we present methods that improve the sampling distribution by systematically adapting it as we obtain information from the samples. We present a stochastic-gradient-descent method for sequentially updating the sampling distribution based on the direct minization of the variance. We also present other stochastic-gradient-descent methods based on the minimizationof typical notions of distance between the current sampling distribution and approximations of the target, optimal distribution. We finally validate and compare the different methods empirically by applying them to the problem of action evaluation in influence diagrams.