Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models

TL;DR

This paper derives the finite-sample and asymptotic distributions of various thresholding estimators in high-dimensional Gaussian regression, considering known and unknown error variance, and analyzes their consistency and convergence rates.

Contribution

It provides the first comprehensive distributional analysis of hard, soft, and adaptive soft-thresholding estimators in high-dimensional settings with diverging parameters.

Findings

Finite-sample distributions are explicitly derived.

Asymptotic behavior and convergence rates are characterized.

Effects of variance estimation on estimator distribution are analyzed.

Abstract

We study the distribution of hard-, soft-, and adaptive soft-thresholding estimators within a linear regression model where the number of parameters k can depend on sample size n and may diverge with n. In addition to the case of known error-variance, we define and study versions of the estimators when the error-variance is unknown. We derive the finite-sample distribution of each estimator and study its behavior in the large-sample limit, also investigating the effects of having to estimate the variance when the degrees of freedom n-k does not tend to infinity or tends to infinity very slowly. Our analysis encompasses both the case where the estimators are tuned to perform consistent model selection and the case where the estimators are tuned to perform conservative model selection. Furthermore, we discuss consistency, uniform consistency and derive the uniform convergence rate under either type of tuning.