Uniform-in-Submodel Bounds for Linear Regression in a Model Free Framework

TL;DR

This paper establishes uniform bounds for linear regression estimators across submodels in a model-free, non-asymptotic framework, aiding interpretation and inference after variable selection in high-dimensional data.

Contribution

It provides the first non-asymptotic, model-free uniform bounds for linear regression estimators over submodels, applicable to both independent and dependent data.

Findings

Uniform bounds hold for estimators over submodels

Results are non-asymptotic and model-free

Applicable to dependent data scenarios

Abstract

For the last two decades, high-dimensional data and methods have proliferated throughout the literature. Yet, the classical technique of linear regression has not lost its usefulness in applications. In fact, many high-dimensional estimation techniques can be seen as variable selection that leads to a smaller set of variables (a ``sub-model'') where classical linear regression applies. We analyze linear regression estimators resulting from model-selection by proving estimation error and linear representation bounds uniformly over sets of submodels. Based on deterministic inequalities, our results provide ``good'' rates when applied to both independent and dependent data. These results are useful in meaningfully interpreting the linear regression estimator obtained after exploring and reducing the variables and also in justifying post model-selection inference. All results are derived under no model assumptions and are non-asymptotic in nature.