Optimal Two-Step Prediction in Regression

TL;DR

This paper proposes a new, efficient, and theoretically sound two-step prediction method for high-dimensional regression that avoids costly cross-validation for tuning parameter calibration.

Contribution

It introduces a novel calibration scheme for variable selection in high-dimensional regression that is easy to implement and offers finite sample guarantees.

Findings

New calibration scheme reduces computational cost

Method provides finite sample guarantees

Improves efficiency over cross-validation

Abstract

High-dimensional prediction typically comprises two steps: variable selection and subsequent least-squares refitting on the selected variables. However, the standard variable selection procedures, such as the lasso, hinge on tuning parameters that need to be calibrated. Cross-validation, the most popular calibration scheme, is computationally costly and lacks finite sample guarantees. In this paper, we introduce an alternative scheme, easy to implement and both computationally and theoretically efficient.