Variable-width confidence intervals in Gaussian regression and penalized maximum likelihood estimators

TL;DR

This paper investigates the properties of variable-width confidence intervals that include penalized maximum likelihood estimators like LASSO and SCAD in Gaussian regression, highlighting their performance under different model selection regimes.

Contribution

It extends previous analysis to variable-width intervals and demonstrates conditions under which these intervals outperform standard ones in sparse regression contexts.

Findings

Variable-width intervals perform poorly compared to standard intervals under consistent model selection.

They perform better when the tuning parameter leads to conservative model selection.

The intervals include penalized estimators for appropriate tuning parameter choices.

Abstract

Hard thresholding, LASSO , adaptive LASSO and SCAD point estimators have been suggested for use in the linear regression context when most of the components of the regression parameter vector are believed to be zero, a sparsity type of assumption. Potscher and Schneider, 2010, Electronic Journal of Statistics, have considered the properties of fixed-width confidence intervals that include one of these point estimators (for all possible data values). They consider a normal linear regression model with orthogonal regressors and show that these confidence intervals are longer than the standard confidence interval (based on the maximum likelihood estimator) when the tuning parameter for these point estimators is chosen to lead to either conservative or consistent model selection. We extend this analysis to the case of variable-width confidence intervals that include one of these point estimators (for all possible data values). In consonance with these findings of Potscher and Schneider, we find that these confidence intervals perform poorly by comparison with the standard confidence interval, when the tuning parameter for these point estimators is chosen to lead to consistent model selection. However, when the tuning parameter for these point estimators is chosen to lead to conservative model selection, our conclusions differ from those of Potscher and Schneider. We consider the variable-width confidence intervals of Farchione and Kabaila, 2008, Statistics & Probability Letters, which have advantages over the standard confidence interval in the context that there is a belief in a sparsity type of assumption. These variable-width confidence intervals are shown to include the hard thresholding, LASSO, adaptive LASSO and SCAD estimators (for all possible data values) provided that the tuning parameters for these estimators are chosen to belong to an appropriate interval.