Heterogeneous transfer learning for high-dimensional regression with feature mismatch

TL;DR

This paper introduces a novel heterogeneous transfer learning method for high-dimensional regression with mismatched features, providing statistical error guarantees and handling both linear and nonparametric feature mappings.

Contribution

It develops a transfer learning approach that learns feature mappings and imputes missing features, with theoretical bounds on estimation and prediction errors in high-dimensional settings.

Findings

Provides upper bounds on estimation errors

Handles both linear and nonparametric feature maps

Demonstrates effectiveness in high-dimensional transfer learning

Abstract

We consider Heterogeneous Transfer Learning (HTL) from a source to a new target domain for high-dimensional regression with differing feature sets. Most homogeneous TL methods assume that target and source domains share the same feature space, which limits their practical applicability. In applications, the target and source features are frequently different due to the inability to measure certain variables in data-poor target environments. Conversely, existing HTL methods do not provide statistical error guarantees, limiting their utility for scientific discovery. Our method first learns a feature map between the missing and observed features, leveraging the vast source data, and then imputes the missing features in the target. Using the combined matched and imputed features, we then perform a two-step transfer learning for penalized regression. We develop upper bounds on estimation and prediction errors, assuming that the source and target parameters differ sparsely but without assuming sparsity in the target model. We obtain results for both when the feature map is linear and when it is nonparametrically specified as unknown functions. Our results elucidate how estimation and prediction errors of HTL depend on the model's complexity, sample size, the quality and differences in feature maps, and differences in the models across domains.