A conditional one-output likelihood formulation for multitask Gaussian processes

TL;DR

This paper introduces a novel multitask Gaussian process method that simplifies modeling by avoiding low-rank approximations, reduces hyperparameters, and improves accuracy and computational efficiency in multioutput regression tasks.

Contribution

The authors propose a new approach that reduces multitask GP modeling to conditioned univariate GPs, eliminating the need for low-rank approximations and hyperparameter validation.

Findings

Accurately recovers original noise and signal matrices.

Outperforms state-of-the-art MTGP methods in accuracy.

Maintains computational competitiveness with existing tools.

Abstract

Multitask Gaussian processes (MTGP) are the Gaussian process (GP) framework's solution for multioutput regression problems in which the $T$ elements of the regressors cannot be considered conditionally independent given the observations. Standard MTGP models assume that there exist both a multitask covariance matrix as a function of an intertask matrix, and a noise covariance matrix. These matrices need to be approximated by a low rank simplification of order $P$ in order to reduce the number of parameters to be learnt from $T^2$ to $TP$. Here we introduce a novel approach that simplifies the multitask learning by reducing it to a set of conditioned univariate GPs without the need for any low rank approximations, therefore completely eliminating the requirement to select an adequate value for hyperparameter $P$. At the same time, by extending this approach with both a hierarchical and an approximate model, the proposed extensions are capable of recovering the multitask covariance and noise matrices after learning only $2T$ parameters, avoiding the validation of any model hyperparameter and reducing the overall complexity of the model as well as the risk of overfitting. Experimental results over synthetic and real problems confirm the advantages of this inference approach in its ability to accurately recover the original noise and signal matrices, as well as the achieved performance improvement in comparison to other state of art MTGP approaches. We have also integrated the model with standard GP toolboxes, showing that it is computationally competitive with state of the art options.