Transfer Learning for High-dimensional Quantile Regression with Distribution Shift

TL;DR

This paper develops a new transfer learning framework for high-dimensional quantile regression that effectively handles various types of distribution shifts, with theoretical guarantees and practical validation.

Contribution

It introduces a novel transferable set and transfer framework for high-dimensional quantile regression under distribution shifts, with theoretical error bounds and inference methods.

Findings

The proposed method achieves accurate estimation under distribution shifts.

The framework demonstrates superior performance in simulations and real data.

Statistical inference is improved with an orthogonal debiased approach.

Abstract

Information from related source studies can often enhance the findings of a target study. However, the distribution shift between target and source studies can severely impact the efficiency of knowledge transfer. In the high-dimensional regression setting, existing transfer approaches mainly focus on the parameter shift. In this paper, we focus on the high-dimensional quantile regression with knowledge transfer under three types of distribution shift: parameter shift, covariate shift, and residual shift. We propose a novel transferable set and a new transfer framework to address the above three discrepancies. Non-asymptotic estimation error bounds and source detection consistency are established to validate the availability and superiority of our method in the presence of distribution shift. Additionally, an orthogonal debiased approach is proposed for statistical inference with knowledge transfer, leading to sharper asymptotic results. Extensive simulation results as well as real data applications further demonstrate the effectiveness of our proposed procedure.