Variational Gaussian Approximation for Poisson Data

TL;DR

This paper develops a variational Gaussian approximation method for Bayesian inference in Poisson models, providing explicit formulas, convergence analysis, and efficient algorithms, with applications to hyperparameter selection and numerical validation.

Contribution

It introduces a novel variational Gaussian approximation framework for Poisson posteriors, including explicit formulas, convergence guarantees, and algorithms for hyperparameter tuning.

Findings

Explicit expression for the lower bound functional.

Proven existence and uniqueness of the optimal Gaussian approximation.

Efficient algorithms with convergence analysis and numerical validation.

Abstract

The Poisson model is frequently employed to describe count data, but in a Bayesian context it leads to an analytically intractable posterior probability distribution. In this work, we analyze a variational Gaussian approximation to the posterior distribution arising from the Poisson model with a Gaussian prior. This is achieved by seeking an optimal Gaussian distribution minimizing the Kullback-Leibler divergence from the posterior distribution to the approximation, or equivalently maximizing the lower bound for the model evidence. We derive an explicit expression for the lower bound, and show the existence and uniqueness of the optimal Gaussian approximation. The lower bound functional can be viewed as a variant of classical Tikhonov regularization that penalizes also the covariance. Then we develop an efficient alternating direction maximization algorithm for solving the optimization problem, and analyze its convergence. We discuss strategies for reducing the computational complexity via low rank structure of the forward operator and the sparsity of the covariance. Further, as an application of the lower bound, we discuss hierarchical Bayesian modeling for selecting the hyperparameter in the prior distribution, and propose a monotonically convergent algorithm for determining the hyperparameter. We present extensive numerical experiments to illustrate the Gaussian approximation and the algorithms.