On a class of high dimensional linear regression methods with debiasing and thresholding

TL;DR

This paper introduces a unified framework for high-dimensional linear regression that includes traditional, novel, and debiased thresholding methods, offering theoretical guarantees and improved feature selection capabilities.

Contribution

It proposes a new class of debiased and thresholded regression methods within a unified framework, with theoretical analysis and practical advantages over existing techniques.

Findings

Methods show consistent estimation in high dimensions

Debiased methods facilitate feature selection

Numerical results demonstrate superior finite sample performance

Abstract

In this paper, we introduce a unified framework, inspired by classical regularization theory, for designing and analyzing a broad class of linear regression approaches. Our framework encompasses traditional methods like least squares regression and Ridge regression, as well as innovative techniques, including seven novel regression methods such as Landweber and Showalter regressions. Within this framework, we further propose a class of debiased and thresholded regression methods to promote feature selection, particularly in terms of sparsity. These methods may offer advantages over conventional regression techniques, including Lasso, due to their ease of computation via a closed-form expression. Theoretically, we establish consistency results and Gaussian approximation theorems for this new class of regularization methods. Extensive numerical simulations further demonstrate that the debiased and thresholded counterparts of linear regression methods exhibit favorable finite sample performance and may be preferable in certain settings.