# Honest confidence sets for high-dimensional regression by projection and shrinkage

## TL;DR

This paper introduces a novel projection and shrinkage method to construct honest confidence sets in high-dimensional linear regression, effectively handling unknown sparsity and achieving optimal adaptation.

## Contribution

It develops a new approach that separates signals into strong and weak groups, ensuring honest confidence sets without sparsity constraints and adapting to the true sparsity.

## Key findings

- Outperforms existing methods in finite sample scenarios.
- Achieves honest confidence sets over the full parameter space.
- Adapts to the optimal rate when the true parameter is sparse.

## Abstract

The issue of honesty in constructing confidence sets arises in nonparametric regression. While optimal rate in nonparametric estimation can be achieved and utilized to construct sharp confidence sets, severe degradation of confidence level often happens after estimating the degree of smoothness. Similarly, for high-dimensional regression, oracle inequalities for sparse estimators could be utilized to construct sharp confidence sets. Yet the degree of sparsity itself is unknown and needs to be estimated, causing the honesty problem. To resolve this issue, we develop a novel method to construct honest confidence sets for sparse high-dimensional linear regression. The key idea in our construction is to separate signals into a strong and a weak group, and then construct confidence sets for each group separately. This is achieved by a projection and shrinkage approach, the latter implemented via Stein estimation and the associated Stein unbiased risk estimate. Our confidence set is honest over the full parameter space without any sparsity constraints, while its diameter adapts to the optimal rate of $n^{-1/4}$ when the true parameter is indeed sparse. Through extensive numerical comparisons, we demonstrate that our method outperforms other competitors with big margins for finite samples, including oracle methods built upon the true sparsity of the underlying model.