Simultaneous Estimation and Dataset Selection for Transfer Learning in High Dimensions by a Non-convex Penalty

TL;DR

This paper introduces a novel approach that simultaneously estimates model parameters and selects informative datasets for high-dimensional transfer learning using a non-convex penalty, improving efficiency and accuracy.

Contribution

It develops a unified framework for joint estimation and dataset selection in transfer learning with high-dimensional data, employing non-convex penalties and advanced optimization algorithms.

Findings

The proposed method effectively identifies useful source datasets.

The algorithms are theoretically justified for both statistical accuracy and computational efficiency.

Numerical experiments validate the method's superior performance.

Abstract

In this paper, we propose to estimate model parameters and identify informative source datasets simultaneously for high-dimensional transfer learning problems with the aid of a non-convex penalty, in contrast to the separate useful dataset selection and transfer learning procedures in the existing literature. To numerically solve the non-convex problem with respect to two specific statistical models, namely the sparse linear regression and the generalized low-rank trace regression models, we adopt the difference of convex (DC) programming with the alternating direction method of multipliers (ADMM) procedures. We theoretically justify the proposed algorithm from both statistical and computational perspectives. Extensive numerical results are reported alongside to validate the theoretical assertions. An \texttt{R} package \texttt{MHDTL} is developed to implement the proposed methods.