Variational inference based on a subclass of closed skew normals

TL;DR

This paper introduces a subclass of closed skew normals for variational inference, enhancing the flexibility and accuracy of posterior approximation in Bayesian models, especially with limited data or prior information.

Contribution

It proposes a new subclass of skew normal distributions for variational inference, with analytic natural gradient updates and solutions for maximum likelihood estimation issues.

Findings

Improved posterior approximation accuracy with skew normals.

Analytic natural gradient updates derived for optimization.

Resolutions for maximum likelihood estimation problems.

Abstract

Gaussian distributions are widely used in Bayesian variational inference to approximate intractable posterior densities, but the ability to accommodate skewness can improve approximation accuracy significantly, when data or prior information is scarce. We study the properties of a subclass of closed skew normals constructed using affine transformation of independent standardized univariate skew normals as the variational density, and illustrate how it provides increased flexibility and accuracy in approximating the joint posterior in various applications, by overcoming limitations in existing skew normal variational approximations. The evidence lower bound is optimized using stochastic gradient ascent, where analytic natural gradient updates are derived. We also demonstrate how problems in maximum likelihood estimation of skew normal parameters occur similarly in stochastic variational inference, and can be resolved using the centered parametrization. Supplemental materials are available online.