Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems

TL;DR

This paper introduces DF-GMVI, a derivative-free variational inference method using Gaussian mixtures, designed for efficient Bayesian inverse problems with complex, multimodal, and high-dimensional posteriors, demonstrated on challenging scientific computing scenarios.

Contribution

It develops a novel derivative-free Gaussian mixture variational inference framework combining Fisher-Rao natural gradient and specialized quadrature, ensuring stability and efficiency in complex Bayesian inverse problems.

Findings

Successfully approximates multimodal and curved distributions in up to 100 dimensions.

Demonstrates practical recovery of initial conditions in Navier-Stokes equations.

Ensures covariance positivity and affine invariance in the variational framework.

Abstract

This paper is concerned with the approximation of probability distributions known up to normalization constants, with a focus on Bayesian inference for large-scale inverse problems in scientific computing. In this context, key challenges include costly repeated evaluations of forward models, multimodality, and inaccessible gradients for the forward model. To address them, we develop a variational inference framework that combines Fisher-Rao natural gradient with specialized quadrature rules to enable derivative free updates of Gaussian mixture variational families. The resulting method, termed Derivative Free Gaussian Mixture Variational Inference (DF-GMVI), guarantees covariance positivity and affine invariance, offering a stable and efficient framework for approximating complex posterior distributions. The effectiveness of DF-GMVI is demonstrated through numerical experiments on challenging scenarios, including distributions with multiple modes, infinitely many modes, and curved modes in spaces with up to 100 dimensions. The method's practicality is further demonstrated in a large-scale application, where it successfully recovers the initial conditions of the Navier-Stokes equations from solution data at positive times.