# Conditionally structured variational Gaussian approximation with importance weights

## TL;DR

This paper introduces a flexible variational inference method for hierarchical models that leverages model structure and importance weights to improve approximation accuracy over traditional Gaussian methods.

## Contribution

It proposes a novel class of variational approximations that utilize model partitioning and importance weighting, enhancing inference in complex hierarchical models.

## Key findings

- Significant improvement over Gaussian variational approximation.
- Effective in generalized linear mixed models and state space models.
- Enhanced estimation of log marginal likelihood.

## Abstract

We develop flexible methods of deriving variational inference for models with complex latent variable structure. By splitting the variables in these models into "global" parameters and "local" latent variables, we define a class of variational approximations that exploit this partitioning and go beyond Gaussian variational approximation. This approximation is motivated by the fact that in many hierarchical models, there are global variance parameters which determine the scale of local latent variables in their posterior conditional on the global parameters. We also consider parsimonious parametrizations by using conditional independence structure, and improved estimation of the log marginal likelihood and variational density using importance weights. These methods are shown to improve significantly on Gaussian variational approximation methods for a similar computational cost. Application of the methodology is illustrated using generalized linear mixed models and state space models.