Source-Function Weighted-Transfer Learning for Nonparametric Regression with Seemingly Similar Sources

TL;DR

This paper introduces a source-function weighted transfer learning method for nonparametric regression that adapts to both similar and dissimilar source-target scenarios, improving estimation efficiency and robustness.

Contribution

It proposes a novel seeming similarity condition and a weighted transfer learning estimator that is effective across various source-target relationships, with theoretical and empirical validation.

Findings

The method achieves high estimation efficiency in diverse scenarios.

It is competitive with full data estimators under homogeneous models.

Simulation and real data show significant improvements over competitors.

Abstract

The homogeneity, or more generally, the similarity between source domains and a target domain seems to be essential to a positive transfer learning. In practice, however, the similarity condition is difficult to check and is often violated. In this paper, instead of the popularly used similarity condition, a seeming similarity is introduced, which is defined by a non-orthogonality together with a smoothness. Such a condition is naturally satisfied under common situations and even implies the dissimilarity in some sense. Based on the seeming similarity together with an $L_2$-adjustment, a source-function weighted-transfer learning estimation (sw-TLE) is constructed. By source-function weighting, an adaptive transfer learning is achieved in the sense that it is applied to similar and dissimilar scenarios with a relatively high estimation efficiency. Particularly, under the case with homogenous source and target models, the sw-TLE even can be competitive with the full data estimator. The hidden relationship between the source-function weighting estimator and the James-Stein estimator is established as well, which reveals the structural reasonability of our methodology. Moreover, the strategy does apply to nonparametric and semiparametric models. The comprehensive simulation studies and real data analysis can illustrate that the new strategy is significantly better than the competitors.