Unified Transfer Learning Models in High-Dimensional Linear Regression

TL;DR

This paper introduces UTrans, an interpretable transfer learning model for high-dimensional linear regression that detects transferable variables and source data, improving estimation accuracy and interpretability.

Contribution

The paper develops a unified transfer learning model with theoretical error bounds and a source detection algorithm, enhancing transfer learning in high-dimensional settings.

Findings

UTrans achieves lower estimation and prediction errors than existing methods.

The source detection algorithm effectively excludes nontransferable data.

Application to US mobility data demonstrates practical utility.

Abstract

Transfer learning plays a key role in modern data analysis when: (1) the target data are scarce but the source data are sufficient; (2) the distributions of the source and target data are heterogeneous. This paper develops an interpretable unified transfer learning model, termed as UTrans, which can detect both transferable variables and source data. More specifically, we establish the estimation error bounds and prove that our bounds are lower than those with target data only. Besides, we propose a source detection algorithm based on hypothesis testing to exclude the nontransferable data. We evaluate and compare UTrans to the existing algorithms in multiple experiments. It is shown that UTrans attains much lower estimation and prediction errors than the existing methods, while preserving interpretability. We finally apply it to the US intergenerational mobility data and compare our proposed algorithms to the classical machine learning algorithms.