Cross-Semantic Transfer Learning for High-Dimensional Linear Regression

TL;DR

This paper introduces CSTL, a transfer learning framework for high-dimensional linear regression that leverages cross-semantic feature similarities, outperforming existing methods in various scenarios.

Contribution

It proposes a novel CSTL framework that captures cross-semantic feature relationships using a weighted fusion penalty and provides theoretical guarantees for its effectiveness.

Findings

CSTL outperforms existing methods in simulations.

CSTL effectively handles cross-semantic feature similarities.

Theoretical analysis confirms oracle properties under mild conditions.

Abstract

Current transfer learning methods for high-dimensional linear regression assume feature alignment across domains, restricting their applicability to semantically matched features. In many real-world scenarios, however, distinct features in the target and source domains can play similar predictive roles, creating a form of cross-semantic similarity. To leverage this broader transferability, we propose the Cross-Semantic Transfer Learning (CSTL) framework. It captures potential relationships by comparing each target coefficient with all source coefficients through a weighted fusion penalty. The weights are derived from the derivative of the SCAD penalty, effectively approximating an ideal weighting scheme that preserves transferable signals while filtering out source-specific noise. For computational efficiency, we implement CSTL using the Alternating Direction Method of Multipliers (ADMM). Theoretically, we establish that under mild conditions, CSTL achieves the oracle estimator with overwhelming probability. Empirical results from simulations and a real-data application confirm that CSTL outperforms existing methods in both cross-semantic and partial signal similarity settings.