Importance sampling squared for Bayesian inference in latent variable models

TL;DR

This paper introduces importance sampling squared (IS2), a Bayesian inference method that uses unbiased likelihood estimates, providing theoretical justification, convergence analysis, and practical guidelines for efficient and accurate posterior inference.

Contribution

It formally justifies the use of importance sampling with estimated likelihoods and offers guidelines for balancing computational cost and accuracy in Bayesian inference.

Findings

IS2 achieves fast and accurate posterior inference.

The method's convergence properties are rigorously analyzed.

Empirical results demonstrate effectiveness in complex models.

Abstract

We consider Bayesian inference by importance sampling when the likelihood is analytically intractable but can be unbiasedly estimated. We refer to this procedure as importance sampling squared (IS2), as we can often estimate the likelihood itself by importance sampling. We provide a formal justification for importance sampling when working with an estimate of the likelihood and study its convergence properties. We analyze the effect of estimating the likelihood on the resulting inference and provide guidelines on how to set up the precision of the likelihood estimate in order to obtain an optimal tradeoff? between computational cost and accuracy for posterior inference on the model parameters. We illustrate the procedure in empirical applications for a generalized multinomial logit model and a stochastic volatility model. The results show that the IS2 method can lead to fast and accurate posterior inference under the optimal implementation.