Transfer Learning for Kernel-based Regression

TL;DR

This paper investigates transfer learning in nonparametric kernel regression, proposing new estimators for known and unknown source scenarios, with theoretical guarantees and validated by experiments.

Contribution

It introduces a novel kernel-based transfer learning method for unknown sources with theoretical analysis and practical validation.

Findings

Proposed a two-step kernel estimator for known sources.

Developed an aggregation algorithm for unknown sources.

Validated effectiveness through synthetic and real data experiments.

Abstract

In recent years, transfer learning has garnered significant attention. Its ability to leverage knowledge from related studies to improve generalization performance in a target study has made it highly appealing. This paper focuses on investigating the transfer learning problem within the context of nonparametric regression over a reproducing kernel Hilbert space. The aim is to bridge the gap between practical effectiveness and theoretical guarantees. We specifically consider two scenarios: one where the transferable sources are known and another where they are unknown. For the known transferable source case, we propose a two-step kernel-based estimator by solely using kernel ridge regression. For the unknown case, we develop a novel method based on an efficient aggregation algorithm, which can automatically detect and alleviate the effects of negative sources. This paper provides the statistical properties of the desired estimators and establishes the minimax rate. Through extensive numerical experiments on synthetic data and real examples, we validate our theoretical findings and demonstrate the effectiveness of our proposed method.