# Mixed Variational Inference

## TL;DR

This paper introduces a novel method combining Laplace approximation and variational inference to better capture posterior correlations and improve approximation quality in Bayesian inference.

## Contribution

It proposes a mixed variational inference approach that retains applicability to non-conjugate models and preserves posterior correlations, overcoming limitations of traditional variational methods.

## Key findings

- Improved approximation over Laplace method.
- Enhanced posterior correlation modeling.
- Numerical experiments show superior performance.

## Abstract

The Laplace approximation has been one of the workhorses of Bayesian inference. It often delivers good approximations in practice despite the fact that it does not strictly take into account where the volume of posterior density lies. Variational approaches avoid this issue by explicitly minimising the Kullback-Leibler divergence DKL between a postulated posterior and the true (unnormalised) logarithmic posterior. However, they rely on a closed form DKL in order to update the variational parameters. To address this, stochastic versions of variational inference have been devised that approximate the intractable DKL with a Monte Carlo average. This approximation allows calculating gradients with respect to the variational parameters. However, variational methods often postulate a factorised Gaussian approximating posterior. In doing so, they sacrifice a-posteriori correlations. In this work, we propose a method that combines the Laplace approximation with the variational approach. The advantages are that we maintain: applicability on non-conjugate models, posterior correlations and a reduced number of free variational parameters. Numerical experiments demonstrate improvement over the Laplace approximation and variational inference with factorised Gaussian posteriors.