Transfer Learning Meets Functional Linear Regression: No Negative Transfer under Posterior Drift

TL;DR

This paper develops a transfer learning method for functional linear regression under posterior drift, ensuring no negative transfer and improving estimation accuracy by leveraging auxiliary data when the relationship between responses and covariates changes.

Contribution

It introduces a novel estimator combining least squares and lasso penalty that avoids negative transfer under posterior drift and proposes an adaptive algorithm for robustness against non-informative sources.

Findings

Estimator performs at least as well as classical methods using only target data.

Method effectively prevents negative transfer even with large model contrast.

Simulation and real data demonstrate the algorithm's robustness and effectiveness.

Abstract

Posterior drift refers to changes in the relationship between responses and covariates while the distributions of the covariates remain unchanged. In this work, we explore functional linear regression under posterior drift with transfer learning. Specifically, we investigate when and how auxiliary data can be leveraged to improve the estimation accuracy of the slope function in the target model when posterior drift occurs. We employ the approximated least square method together with a lasso penalty to construct an estimator that transfers beneficial knowledge from source data. Theoretical analysis indicates that our method avoids negative transfer under posterior drift, even when the contrast between slope functions is quite large. Specifically, the estimator is shown to perform at least as well as the classical estimator using only target data, and it enhances the learning of the target model when the source and target models are sufficiently similar. Furthermore, to address scenarios where covariate distributions may change, we propose an adaptive algorithm using aggregation techniques. This algorithm is robust against non-informative source samples and effectively prevents negative transfer. Simulation and real data examples are provided to demonstrate the effectiveness of the proposed algorithm.