$\chi^2$-confidence sets in high-dimensional regression

TL;DR

This paper develops a method for constructing confidence sets for groups of regression coefficients in high-dimensional models using a one-step procedure with square-root Lasso, resulting in an asymptotic chi-squared distribution.

Contribution

It introduces a novel one-step procedure combining square-root Lasso and a multivariate version for confidence set construction in high-dimensional regression.

Findings

The procedure yields an asymptotically chi-squared distributed pivot.

The method provides consistent variance estimation under sparsity.

Sharp oracle inequalities demonstrate small remainder terms under certain conditions.

Abstract

We study a high-dimensional regression model. Aim is to construct a confidence set for a given group of regression coefficients, treating all other regression coefficients as nuisance parameters. We apply a one-step procedure with the square-root Lasso as initial estimator and a multivariate square-root Lasso for constructing a surrogate Fisher information matrix. The multivariate square-root Lasso is based on nuclear norm loss with $\ell_1$-penalty. We show that this procedure leads to an asymptotically $\chi^2$-distributed pivot, with a remainder term depending only on the $\ell_1$-error of the initial estimator. We show that under $\ell_1$-sparsity conditions on the regression coefficients $\beta^0$ the square-root Lasso produces to a consistent estimator of the noise variance and we establish sharp oracle inequalities which show that the remainder term is small under further sparsity conditions on $\beta^0$ and compatibility conditions on the design.