Variational Bayes with Synthetic Likelihood

TL;DR

This paper introduces a variational Bayes approach using synthetic likelihoods for likelihood-free inference, significantly reducing computational costs and enabling application to high-dimensional problems.

Contribution

It develops stochastic gradient variational inference methods tailored for synthetic likelihoods, improving efficiency over traditional MCMC and ABC techniques.

Findings

Method reduces computational overhead compared to MCMC.

Applicable to high-dimensional parameter and summary statistic spaces.

Outperforms existing likelihood-free variational inference methods.

Abstract

Synthetic likelihood is an attractive approach to likelihood-free inference when an approximately Gaussian summary statistic for the data, informative for inference about the parameters, is available. The synthetic likelihood method derives an approximate likelihood function from a plug-in normal density estimate for the summary statistic, with plug-in mean and covariance matrix obtained by Monte Carlo simulation from the model. In this article, we develop alternatives to Markov chain Monte Carlo implementations of Bayesian synthetic likelihoods with reduced computational overheads. Our approach uses stochastic gradient variational inference methods for posterior approximation in the synthetic likelihood context, employing unbiased estimates of the log likelihood. We compare the new method with a related likelihood free variational inference technique in the literature, while at the same time improving the implementation of that approach in a number of ways. These new algorithms are feasible to implement in situations which are challenging for conventional approximate Bayesian computation (ABC) methods, in terms of the dimensionality of the parameter and summary statistic.