Linear shrinkage for predicting responses in large-scale multivariate linear regression

TL;DR

This paper introduces a computationally efficient, tuning-free linear shrinkage method for large-scale multivariate linear regression, outperforming ordinary least squares without requiring structural assumptions.

Contribution

It proposes a novel, scalable shrinkage approach for multivariate regression that avoids parameter tuning and performs well in high-dimensional, large-scale settings.

Findings

Outperforms ordinary least squares asymptotically

Computationally efficient and tuning-free

Effective in high-dimensional, large-scale data

Abstract

We propose a new prediction method for multivariate linear regression problems where the number of features is less than the sample size but the number of outcomes is extremely large. Many popular procedures, such as penalized regression procedures, require parameter tuning that is computationally untenable in such large-scale problems. We take a different approach, motivated by ideas from simultaneous estimation problems, that performs linear shrinkage on ordinary least squares parameter estimates. Our approach is extremely computationally efficient and tuning-free. We show that it can asymptotically outperform ordinary least squares without any structural assumptions on the true regression coefficients and illustrate its good performance in simulations and an analysis of single-cell RNA-seq data.