Learning Multi-Task Gaussian Process Over Heterogeneous Input Domains

TL;DR

This paper introduces a novel multi-task Gaussian process model capable of handling heterogeneous input domains, enabling effective knowledge transfer across tasks with varying feature spaces, validated on diverse applications.

Contribution

It proposes the HSVLMC model with a stochastic variational framework that aligns heterogeneous input domains and leverages prior domain mappings for improved multi-task learning.

Findings

Outperforms existing LMC models on heterogeneous multi-task cases

Effective input alignment through dimensionality reduction and domain mappings

Validated on multi-fidelity steam turbine exhaust problem

Abstract

Multi-task Gaussian process (MTGP) is a well-known non-parametric Bayesian model for learning correlated tasks effectively by transferring knowledge across tasks. But current MTGPs are usually limited to the multi-task scenario defined in the same input domain, leaving no space for tackling the heterogeneous case, i.e., the features of input domains vary over tasks. To this end, this paper presents a novel heterogeneous stochastic variational linear model of coregionalization (HSVLMC) model for simultaneously learning the tasks with varied input domains. Particularly, we develop the stochastic variational framework with Bayesian calibration that (i) takes into account the effect of dimensionality reduction raised by domain mappings in order to achieve effective input alignment; and (ii) employs a residual modeling strategy to leverage the inductive bias brought by prior domain mappings for better model inference. Finally, the superiority of the proposed model against existing LMC models has been extensively verified on diverse heterogeneous multi-task cases and a practical multi-fidelity steam turbine exhaust problem.