Non-Asymptotic Uncertainty Quantification in High-Dimensional Learning

TL;DR

This paper introduces a data-driven method for non-asymptotic uncertainty quantification in high-dimensional regression and neural networks, addressing bias correction to produce more reliable confidence intervals in finite data settings.

Contribution

It develops a novel bias correction technique for debiased LASSO and neural networks, enabling accurate non-asymptotic confidence intervals in high-dimensional learning.

Findings

Provides a data-driven bias adjustment method

Extends non-asymptotic UQ to neural networks

Improves confidence interval accuracy in finite samples

Abstract

Uncertainty quantification (UQ) is a crucial but challenging task in many high-dimensional regression or learning problems to increase the confidence of a given predictor. We develop a new data-driven approach for UQ in regression that applies both to classical regression approaches such as the LASSO as well as to neural networks. One of the most notable UQ techniques is the debiased LASSO, which modifies the LASSO to allow for the construction of asymptotic confidence intervals by decomposing the estimation error into a Gaussian and an asymptotically vanishing bias component. However, in real-world problems with finite-dimensional data, the bias term is often too significant to be neglected, resulting in overly narrow confidence intervals. Our work rigorously addresses this issue and derives a data-driven adjustment that corrects the confidence intervals for a large class of predictors by estimating the means and variances of the bias terms from training data, exploiting high-dimensional concentration phenomena. This gives rise to non-asymptotic confidence intervals, which can help avoid overestimating uncertainty in critical applications such as MRI diagnosis. Importantly, our analysis extends beyond sparse regression to data-driven predictors like neural networks, enhancing the reliability of model-based deep learning. Our findings bridge the gap between established theory and the practical applicability of such debiased methods.