Bayesian Covariance Modelling of Large Tensor-Variate Data Sets $\&$ Inverse Non-parametric Learning of the Unknown Model Parameter Vector

TL;DR

This paper introduces a Bayesian covariance modeling approach for large tensor-variate datasets using tensor-normal distributions and efficient MCMC sampling, enabling the learning of unknown model parameters from high-dimensional data.

Contribution

It presents a novel Bayesian method employing tensor-normal models and Transformation-based MCMC for efficient covariance estimation in high-dimensional tensor data.

Findings

Effective covariance modeling of large tensor datasets.

Successful inference of unknown model parameters.

Enhanced computational efficiency in high-dimensional Bayesian inference.

Abstract

Tensor-valued data are being encountered increasingly more commonly, in the biological, natural as well as the social sciences. The learning of the unknown model parameter vector given such data, involves covariance modelling of such data, though this can be difficult owing to the high-dimensional nature of the data, where the numerical challenge of such modelling can only be compounded by the largeness of the available data set. Assuming such data to be modelled using a correspondingly high-dimensional Gaussian Process (${\cal GP}$), the joint density of a finite set of such data sets is then a tensor normal distribution, with density parametrised by a mean tensor $\boldsymbol{M}$ (that is of the same dimensionality as the $k$-tensor valued observable), and the $k$ covariance matrices $\boldsymbol{\Sigma}_1,...,\boldsymbol{\Sigma}_k$. When aiming to model the covariance structure of the data, we need to estimate/learn $\{\boldsymbol{\Sigma}_1,...,\boldsymbol{\Sigma}_k \}$ and $\boldsymbol{M}$, given tha data. We present a new method in which we perform such covariance modelling by first expressing the probability density of the available data sets as tensor-normal. We then invoke appropriate priors on these unknown parameters and express the posterior of the unknowns given the data. We sample from this posterior using an appropriate variant of Metropolis Hastings. Since the classical MCMC is time and resource intensive in high-dimensional state spaces, we use an efficient variant of the Metropolis-Hastings algorithm--Transformation based MCMC--employed to perform efficient sampling from a high-dimensional state space. Once we perform the covariance modelling of such a data set, we will learn the unknown model parameter vector at which a measured (or test) data set has been obtained, given the already modelled data (training data), augmented by the test data.