Unbiased MLMC-based variational Bayes for likelihood-free inference

TL;DR

This paper introduces unbiased MLMC-based gradient estimators for variational Bayes in likelihood-free inference, incorporating RQMC sampling to improve convergence and speed, with theoretical guarantees and empirical validation.

Contribution

It presents a novel unbiased MLMC approach for VB with intractable likelihoods, integrating RQMC for enhanced convergence and efficiency, unlike biased existing methods.

Findings

RQMC accelerates convergence of the VB algorithm.

The proposed method achieves better parameter estimates.

Theoretical guarantees support the method's validity.

Abstract

Variational Bayes (VB) is a popular tool for Bayesian inference in statistical modeling. Recently, some VB algorithms are proposed to handle intractable likelihoods with applications such as approximate Bayesian computation. In this paper, we propose several unbiased estimators based on multilevel Monte Carlo (MLMC) for the gradient of Kullback-Leibler divergence between the posterior distribution and the variational distribution when the likelihood is intractable, but can be estimated unbiasedly. The new VB algorithm differs from the VB algorithms in the literature which usually render biased gradient estimators. Moreover, we incorporate randomized quasi-Monte Carlo (RQMC) sampling within the MLMC-based gradient estimators, which was known to provide a favorable rate of convergence in numerical integration. Theoretical guarantees for RQMC are provided in this new setting. Numerical experiments show that using RQMC in MLMC greatly speeds up the VB algorithm, and finds a better parameter value than some existing competitors do.