Consistency of Importance Sampling estimates based on dependent sample sets and an application to models with factorizing likelihoods

TL;DR

This paper proves that importance sampling estimates remain consistent even with dependent samples and introduces Sample Inflation, a method to reduce variance in Bayesian models with factorizing likelihoods, demonstrated on Gaussian and mixture models.

Contribution

It establishes the consistency of importance sampling with dependent samples and proposes Sample Inflation to improve variance reduction in Bayesian inference.

Findings

Sample Inflation reduces variance in importance sampling.

Consistency of importance sampling holds with dependent sample sets.

Effective on Gaussian and mixture model examples.

Abstract

In this paper, I proof that Importance Sampling estimates based on dependent sample sets are consistent under certain conditions. This can be used to reduce variance in Bayesian Models with factorizing likelihoods, using sample sets that are much larger than the number of likelihood evaluations, a technique dubbed Sample Inflation. I evaluate Sample Inflation on a toy Gaussian problem and two Mixture Models.