Deep Bayesian Supervised Learning given Hypercuboidally-shaped, Discontinuous Data, using Compound Tensor-Variate & Scalar-Variate Gaussian Processes

TL;DR

This paper develops a Bayesian deep learning framework using tensor- and scalar-variate Gaussian processes to model high-dimensional, discontinuous, hypercuboidally-shaped data, enabling inverse prediction and model validation.

Contribution

It introduces a novel deep Bayesian approach with non-stationary covariance kernels modeled as functions of tensor GP samples, suitable for discontinuous, high-dimensional data.

Findings

Successfully modeled hypercuboidally-shaped, discontinuous data

Demonstrated effective inverse prediction of system parameters

Validated model with real dataset and forward prediction

Abstract

We undertake Bayesian learning of the high-dimensional functional relationship between a system parameter vector and an observable, that is in general tensor-valued. The ultimate aim is Bayesian inverse prediction of the system parameters, at which test data is recorded. We attempt such learning given hypercuboidally-shaped data that displays strong discontinuities, rendering learning challenging. We model the sought high-dimensional function, with a tensor-variate Gaussian Process (GP), and use three independent ways for learning covariance matrices of the resulting likelihood, which is Tensor-Normal. We demonstrate that the discontinuous data demands that implemented covariance kernels be non-stationary--achieved by modelling each kernel hyperparameter, as a function of the sample function of the invoked tensor-variate GP. Each such function can be shown to be temporally-evolving, and treated as a realisation from a distinct scalar-variate GP, with covariance described adaptively by collating information from a historical set of samples of chosen sample-size. We prove that deep-learning using 2-"layers", suffice, where the outer-layer comprises the tensor-variate GP, compounded with multiple scalar-variate GPs in the "inner-layer", and undertake inference with Metropolis-within-Gibbs. We apply our method to a cuboidally-shaped, discontinuous, real dataset, and subsequently perform forward prediction to generate data from our model, given our results--to perform model-checking.