Flexible Variational Bayes based on a Copula of a Mixture

TL;DR

This paper introduces a flexible variational Bayes method using copulas of mixtures, combining boosting, natural gradient, and variance reduction to efficiently approximate complex posterior distributions in Bayesian models.

Contribution

It develops a novel variational approximation framework based on copulas of mixtures, enhancing flexibility and efficiency over traditional methods.

Findings

Effective in approximating multimodal, skewed, and heavy-tailed posteriors

Demonstrated on simulated and real datasets

Applied successfully to Bayesian neural network regression

Abstract

Variational Bayes methods approximate the posterior density by a family of tractable distributions whose parameters are estimated by optimisation. Variational approximation is useful when exact inference is intractable or very costly. Our article develops a flexible variational approximation based on a copula of a mixture, which is implemented by combining boosting, natural gradient, and a variance reduction method. The efficacy of the approach is illustrated by using simulated and real datasets to approximate multimodal, skewed and heavy-tailed posterior distributions, including an application to Bayesian deep feedforward neural network regression models.