A Geometric Variational Approach to Bayesian Inference

TL;DR

This paper introduces a Riemannian geometric framework for Bayesian variational inference using the Fisher-Rao metric, enabling tighter bounds and a novel gradient algorithm on the probability density manifold.

Contribution

It develops a geometric variational inference method based on the Fisher-Rao metric and the Hilbert sphere, providing tighter bounds and a new optimization algorithm.

Findings

Demonstrates improved posterior approximation bounds.

Validates the approach with simulations and real data.

Introduces a gradient algorithm leveraging Riemannian geometry.

Abstract

We propose a novel Riemannian geometric framework for variational inference in Bayesian models based on the nonparametric Fisher-Rao metric on the manifold of probability density functions. Under the square-root density representation, the manifold can be identified with the positive orthant of the unit hypersphere in L2, and the Fisher-Rao metric reduces to the standard L2 metric. Exploiting such a Riemannian structure, we formulate the task of approximating the posterior distribution as a variational problem on the hypersphere based on the alpha-divergence. This provides a tighter lower bound on the marginal distribution when compared to, and a corresponding upper bound unavailable with, approaches based on the Kullback-Leibler divergence. We propose a novel gradient-based algorithm for the variational problem based on Frechet derivative operators motivated by the geometry of the Hilbert sphere, and examine its properties. Through simulations and real-data applications, we demonstrate the utility of the proposed geometric framework and algorithm on several Bayesian models.