Transfer Learning with Uncertainty Quantification: Random Effect Calibration of Source to Target (RECaST)

TL;DR

RECaST introduces a novel transfer learning framework that uses a Cauchy random effect for model recalibration, providing reliable uncertainty quantification across diverse models and populations.

Contribution

The paper presents RECaST, a flexible, model-agnostic transfer learning method that incorporates uncertainty quantification through a Cauchy random effect, validated for linear and nonlinear models.

Findings

RECaST achieves nominal coverage in prediction sets for linear models.

The method is robust to asymptotic approximations in nonlinear models.

RECaST performs well in simulations and real hospital data applications.

Abstract

Transfer learning uses a data model, trained to make predictions or inferences on data from one population, to make reliable predictions or inferences on data from another population. Most existing transfer learning approaches are based on fine-tuning pre-trained neural network models, and fail to provide crucial uncertainty quantification. We develop a statistical framework for model predictions based on transfer learning, called RECaST. The primary mechanism is a Cauchy random effect that recalibrates a source model to a target population; we mathematically and empirically demonstrate the validity of our RECaST approach for transfer learning between linear models, in the sense that prediction sets will achieve their nominal stated coverage, and we numerically illustrate the method's robustness to asymptotic approximations for nonlinear models. Whereas many existing techniques are built on particular source models, RECaST is agnostic to the choice of source model. For example, our RECaST transfer learning approach can be applied to a continuous or discrete data model with linear or logistic regression, deep neural network architectures, etc. Furthermore, RECaST provides uncertainty quantification for predictions, which is mostly absent in the literature. We examine our method's performance in a simulation study and in an application to real hospital data.