On the use of bootstrap with variational inference: Theory, interpretation, and a two-sample test example

TL;DR

This paper explores the integration of bootstrap methods with variational inference to quantify uncertainty, providing theoretical insights and demonstrating a two-sample test for functional disability data.

Contribution

It develops two bootstrap approaches for uncertainty quantification in variational inference and establishes their theoretical foundations in different dimensions.

Findings

Bootstrap methods effectively quantify uncertainty in variational inference.

Theoretical validation of bootstrap approaches in fixed and increasing dimensions.

Application to real data demonstrates practical utility in change detection.

Abstract

Variational inference is a general approach for approximating complex density functions, such as those arising in latent variable models, popular in machine learning. It has been applied to approximate the maximum likelihood estimator and to carry out Bayesian inference, however, quantification of uncertainty with variational inference remains challenging from both theoretical and practical perspectives. This paper is concerned with developing uncertainty measures for variational inference by using bootstrap procedures. We first develop two general bootstrap approaches for assessing the uncertainty of a variational estimate and the study the underlying bootstrap theory in both fixed- and increasing-dimension settings. We then use the bootstrap approach and our theoretical results in the context of mixed membership modeling with multivariate binary data on functional disability from the National Long Term Care Survey. We carry out a two-sample approach to test for changes in the repeated measures of functional disability for the subset of individuals present in 1989 and 1994 waves.