Robust and flexible learning of a high-dimensional classification rule using auxiliary outcomes

TL;DR

This paper introduces a robust transfer learning method for high-dimensional classification that leverages auxiliary outcomes to improve estimation accuracy, correcting biases from traditional multi-task learning.

Contribution

It develops a novel two-step approach combining multi-task learning with a calibration step to reduce bias and enhance estimation of the target outcome in high-dimensional settings.

Findings

The method achieves lower estimation error than single-outcome models.

Simulations demonstrate improved accuracy over traditional multi-task learning.

Real data analysis confirms the method's practical effectiveness.

Abstract

Correlated outcomes are common in many practical problems. In some settings, one outcome is of particular interest, and others are auxiliary. To leverage information shared by all the outcomes, traditional multi-task learning (MTL) minimizes an averaged loss function over all the outcomes, which may lead to biased estimation for the target outcome, especially when the MTL model is mis-specified. In this work, based on a decomposition of estimation bias into two types, within-subspace and against-subspace, we develop a robust transfer learning approach to estimating a high-dimensional linear decision rule for the outcome of interest with the presence of auxiliary outcomes. The proposed method includes an MTL step using all outcomes to gain efficiency, and a subsequent calibration step using only the outcome of interest to correct both types of biases. We show that the final estimator can achieve a lower estimation error than the one using only the single outcome of interest. Simulations and real data analysis are conducted to justify the superiority of the proposed method.