tvGP-VAE: Tensor-variate Gaussian Process Prior Variational Autoencoder

TL;DR

The paper introduces tvGP-VAE, a novel variational autoencoder that employs tensor-variate Gaussian process priors to explicitly model complex correlation structures in high-dimensional tensor data, improving reconstruction performance.

Contribution

It extends VAEs by incorporating tensor-variate Gaussian processes as priors, enabling explicit modeling of correlations in tensor-valued latent variables.

Findings

Model effectively captures correlation structures in tensor data.

Explicit correlation modeling improves reconstruction accuracy.

Tensor-variate GP prior enhances representation of spatiotemporal data.

Abstract

Variational autoencoders (VAEs) are a powerful class of deep generative latent variable model for unsupervised representation learning on high-dimensional data. To ensure computational tractability, VAEs are often implemented with a univariate standard Gaussian prior and a mean-field Gaussian variational posterior distribution. This results in a vector-valued latent variables that are agnostic to the original data structure which might be highly correlated across and within multiple dimensions. We propose a tensor-variate extension to the VAE framework, the tensor-variate Gaussian process prior variational autoencoder (tvGP-VAE), which replaces the standard univariate Gaussian prior and posterior distributions with tensor-variate Gaussian processes. The tvGP-VAE is able to explicitly model correlation structures via the use of kernel functions over the dimensions of tensor-valued latent variables. Using spatiotemporally correlated image time series as an example, we show that the choice of which correlation structures to explicitly represent in the latent space has a significant impact on model performance in terms of reconstruction.