R^2-HGP: A Double-Regularized Gaussian Process for Heterogeneous Transfer Learning

TL;DR

This paper introduces R^2-HGP, a novel Gaussian process framework that effectively handles heterogeneous input spaces, incorporates physical knowledge, and adaptively selects informative sources for transfer learning, improving performance in complex scenarios.

Contribution

The paper proposes a unified double-regularized Gaussian process model with input alignment, physical regularization, and source selection, addressing key challenges in heterogeneous transfer learning.

Findings

Outperforms state-of-the-art benchmarks in simulations.

Effectively incorporates physical knowledge into transfer learning.

Demonstrates robustness and adaptability across diverse real-world cases.

Abstract

Multi-output Gaussian process (MGP) models have attracted significant attention for their flexibility and uncertainty-quantification capabilities, and have been widely adopted in multi-source transfer learning scenarios due to their ability to capture inter-task correlations. However, they still face several challenges in transfer learning. First, the input spaces of the source and target domains are often heterogeneous, which makes direct knowledge transfer difficult. Second, potential prior knowledge and physical information are typically ignored during heterogeneous transfer, hampering the utilization of domain-specific insights and leading to unstable mappings. Third, inappropriate information sharing among target and sources can easily lead to negative transfer. Traditional models fail to address these issues in a unified way. To overcome these limitations, this paper proposes a Double-Regularized Heterogeneous Gaussian Process framework (R^2-HGP). Specifically, a trainable prior probability mapping model is first proposed to align the heterogeneous input domains. The resulting aligned inputs are treated as latent variables, upon which a multi-source transfer GP model is constructed and the entire structure is integrated into a novel conditional variational autoencoder (CVAE) based framework. Physical insights is further incorporated as a regularization term to ensure that the alignment results adhere to known physical knowledge. Next, within the multi-source transfer GP model, a sparsity penalty is imposed on the transfer coefficients, enabling the model to adaptively select the most informative source outputs and suppress negative transfer. Extensive simulations and real-world engineering case studies validate the effectiveness of our R^2-HGP, demonstrating consistent superiority over state-of-the-art benchmarks across diverse evaluation metrics.