Robust Inference with Variational Bayes

TL;DR

This paper develops a method to assess the robustness of Bayesian inferences made using variational Bayes, specifically mean-field variational Bayes, by deriving closed-form sensitivity measures to prior choices, facilitating practical robustness analysis.

Contribution

It introduces a novel approach to compute local prior robustness measures directly within mean-field variational Bayes, overcoming computational challenges of traditional methods.

Findings

Derived closed-form sensitivity measures for mean-field variational Bayes

Applied robustness analysis to a meta-analysis of microcredit interventions

Demonstrated practical utility of the method in complex models

Abstract

In Bayesian analysis, the posterior follows from the data and a choice of a prior and a likelihood. One hopes that the posterior is robust to reasonable variation in the choice of prior and likelihood, since this choice is made by the modeler and is necessarily somewhat subjective. Despite the fundamental importance of the problem and a considerable body of literature, the tools of robust Bayes are not commonly used in practice. This is in large part due to the difficulty of calculating robustness measures from MCMC draws. Although methods for computing robustness measures from MCMC draws exist, they lack generality and often require additional coding or computation. In contrast to MCMC, variational Bayes (VB) techniques are readily amenable to robustness analysis. The derivative of a posterior expectation with respect to a prior or data perturbation is a measure of local robustness to the prior or likelihood. Because VB casts posterior inference as an optimization problem, its methodology is built on the ability to calculate derivatives of posterior quantities with respect to model parameters, even in very complex models. In the present work, we develop local prior robustness measures for mean-field variational Bayes(MFVB), a VB technique which imposes a particular factorization assumption on the variational posterior approximation. We start by outlining existing local prior measures of robustness. Next, we use these results to derive closed-form measures of the sensitivity of mean-field variational posterior approximation to prior specification. We demonstrate our method on a meta-analysis of randomized controlled interventions in access to microcredit in developing countries.