Transfer Learning with Random Coefficient Ridge Regression

TL;DR

This paper introduces transfer learning methods for random coefficient ridge regression, leveraging source models to improve target predictions, with theoretical risk analysis and practical benefits demonstrated in polygenic risk score prediction.

Contribution

It develops optimal weighting schemes for combining target and source ridge estimates in transfer learning, with explicit risk formulas derived via random matrix theory.

Findings

Optimal weights improve prediction accuracy.

Theoretical risks match empirical results.

Application reduces prediction errors in genetic traits.

Abstract

Ridge regression with random coefficients provides an important alternative to fixed coefficients regression in high dimensional setting when the effects are expected to be small but not zeros. This paper considers estimation and prediction of random coefficient ridge regression in the setting of transfer learning, where in addition to observations from the target model, source samples from different but possibly related regression models are available. The informativeness of the source model to the target model can be quantified by the correlation between the regression coefficients. This paper proposes two estimators of regression coefficients of the target model as the weighted sum of the ridge estimates of both target and source models, where the weights can be determined by minimizing the empirical estimation risk or prediction risk. Using random matrix theory, the limiting values of the optimal weights are derived under the setting when $p/n \rightarrow \gamma$, where $p$ is the number of the predictors and $n$ is the sample size, which leads to an explicit expression of the estimation or prediction risks. Simulations show that these limiting risks agree very well with the empirical risks. An application to predicting the polygenic risk scores for lipid traits shows such transfer learning methods lead to smaller prediction errors than the single sample ridge regression or Lasso-based transfer learning.