Skew-symmetric approximations of posterior distributions

TL;DR

This paper introduces a general, provably optimal method to incorporate skewness into symmetric posterior approximations, improving accuracy and convergence rates without additional optimization complexity.

Contribution

It proposes a universal perturbation scheme to enhance symmetric approximations with skewness, applicable to any posterior distribution, and provides theoretical guarantees of improved accuracy and convergence.

Findings

Enhances Gaussian approximations with skewness for better accuracy.

Provably improves convergence rate by at least a sqrt(n) factor.

Numerical studies demonstrate improved approximation quality.

Abstract

Popular deterministic approximations of posterior distributions from, e.g. the Laplace method, variational Bayes and expectation-propagation, generally rely on symmetric approximating families, often taken to be Gaussian. This choice facilitates optimization and inference, but typically affects the quality of the overall approximation. In fact, even in basic parametric models, the posterior distribution often displays asymmetries that yield bias and a reduced accuracy when considering symmetric approximations. Recent research has moved towards more flexible approximating families which incorporate skewness. However, current solutions are often model specific, lack a general supporting theory, increase the computational complexity of the optimization problem, and do not provide a broadly applicable solution to incorporate skewness in any symmetric approximation. This article addresses such a gap by introducing a general and provably optimal strategy to perturb any off-the-shelf symmetric approximation of a generic posterior distribution. This novel perturbation scheme is derived without additional optimization steps, and yields a similarly tractable approximation within the class of skew-symmetric densities that provably enhances the finite sample accuracy of the original symmetric counterpart. Furthermore, under suitable assumptions, it improves the convergence rate to the exact posterior by at least a $\sqrt{n}$ factor, in asymptotic regimes. These advancements are illustrated in numerical studies focusing on skewed perturbations of state-of-the-art Gaussian approximations.