Leave-one-out prediction intervals in linear regression models with many variables

TL;DR

This paper develops and validates leave-one-out prediction intervals for high-dimensional linear regression models, ensuring reliable predictive inference across various estimators and minimal assumptions.

Contribution

It establishes the uniform asymptotic validity of leave-one-out prediction intervals in high-dimensional settings with minimal assumptions on errors and design.

Findings

Intervals are valid for a wide class of linear predictors.

Reliable predictive inference is achievable even with high-dimensional data.

The method is robust to different estimators like LASSO and M-estimators.

Abstract

We study prediction intervals based on leave-one-out residuals in a linear regression model where the number of explanatory variables can be large compared to sample size. We establish uniform asymptotic validity (conditional on the training sample) of the proposed interval under minimal assumptions on the unknown error distribution and the high dimensional design. Our intervals are generic in the sense that they are valid for a large class of linear predictors used to obtain a point forecast, such as robust M-estimators, James-Stein type estimators and penalized estimators like the LASSO. These results show that despite the serious problems of resampling procedures for inference on the unknown parameters, leave-one-out methods can be successfully applied to obtain reliable predictive inference even in high dimensions.