On estimation of the noise variance in high-dimensional linear models

TL;DR

This paper proposes a spectral regularization-based method for accurately estimating noise variance in high-dimensional linear models, utilizing adaptive normalization and concentration bounds.

Contribution

It introduces a novel spectral regularization approach for noise variance estimation in high-dimensional regression, with theoretical guarantees.

Findings

The method achieves tight concentration around the ideal estimator.

Spectral regularization effectively estimates nuisance parameters.

Theoretical bounds validate the estimator's accuracy.

Abstract

We consider the problem of recovering the unknown noise variance in the linear regression model. To estimate the nuisance (a vector of regression coefficients) we use a family of spectral regularisers of the maximum likelihood estimator. The noise estimation is based on the adaptive normalisation of the squared error. We derive the upper bound for the concentration of the proposed method around the ideal estimator (the case of zero nuisance).