Mixing Least-Squares Estimators when the Variance is Unknown

TL;DR

This paper introduces a method for Gaussian regression with unknown variance by mixing least-squares estimators, resulting in adaptive shrinkage estimators with non-asymptotic risk bounds across different statistical models.

Contribution

It presents a novel estimator mixing least-squares methods for unknown variance scenarios, applicable in linear regression and Besov space estimation.

Findings

Estimator acts as a simple shrinkage method in some cases

Provides non-asymptotic risk bounds for the estimator

Applicable in various statistical settings such as linear regression and Besov spaces

Abstract

We propose a procedure to handle the problem of Gaussian regression when the variance is unknown. We mix least-squares estimators from various models according to a procedure inspired by that of Leung and Barron (2007). We show that in some cases the resulting estimator is a simple shrinkage estimator. We then apply this procedure in various statistical settings such as linear regression or adaptive estimation in Besov spaces. Our results provide non-asymptotic risk bounds for the Euclidean risk of the estimator.