A note on the minimax risk of sparse linear regression

TL;DR

This paper precisely characterizes the asymptotic minimax risk for sparse linear regression in high-dimensional settings, providing a sharp formula under certain sparsity and sample size conditions.

Contribution

It derives an asymptotically exact formula for the minimax risk in sparse linear regression with Gaussian design, filling a key gap in high-dimensional statistics literature.

Findings

Minimax risk asymptotically equals 2σ²k/n log(p/k)

Provides a sharp characterization under (k log p)/n → 0

Summarizes existing results and open problems

Abstract

Sparse linear regression is one of the classical and extensively studied problems in high-dimensional statistics and compressed sensing. Despite the substantial body of literature dedicated to this problem, the precise determination of its minimax risk remains elusive. This paper aims to fill this gap by deriving asymptotically constant-sharp characterization for the minimax risk of sparse linear regression. More specifically, the paper focuses on scenarios where the sparsity level, denoted as k, satisfies the condition $(k \log p)/n {\to} 0$, with p and n representing the number of features and observations respectively. We establish that the minimax risk under isotropic Gaussian random design is asymptotically equal to $2{\sigma}^2k/n log(p/k)$, where ${\sigma}$ denotes the standard deviation of the noise. In addition to this result, we will summarize the existing results in the literature, and mention some of the fundamental problems that have still remained open.