Geodesic Variational Bayes for Multiway Covariances

TL;DR

This paper develops a geometric optimization framework for variational approximations of Gaussian multiway array covariances, improving efficiency and accuracy over traditional methods while maintaining interpretability.

Contribution

It introduces a differential geometric approach on the space of covariances, demonstrating superior joint approximation over mean-field methods for multiway Gaussian models.

Findings

Joint approximation outperforms mean-field in efficiency and accuracy.

The method provides a better unstructured Inverse-Wishart posterior approximation.

Efficient gradient expressions are derived for optimization.

Abstract

This article explores the optimization of variational approximations for posterior covariances of Gaussian multiway arrays. To achieve this, we establish a natural differential geometric optimization framework on the space using the pullback of the affine-invariant metric. In the case of a truly separable covariance, we demonstrate a joint approximation in the multiway space outperforms a mean-field approximation in optimization efficiency and provides a superior approximation to an unstructured Inverse-Wishart posterior under the average Mahalanobis distance of the data while maintaining a multiway interpretation. We moreover establish efficient expressions for the Euclidean and Riemannian gradients in both cases of the joint and mean-field approximation. We end with an analysis of commodity trade data.