On universal oracle inequalities related to high-dimensional linear models

TL;DR

This paper develops new data-driven spectral regularization techniques with penalties based on excess risk balancing, providing sharp oracle inequalities for high-dimensional linear models with ill-posed operators.

Contribution

It introduces novel penalties for spectral regularization, enabling data-driven parameter choice with proven sharp oracle inequalities in high-dimensional, ill-posed linear models.

Findings

Derived sharp oracle inequalities for spectral regularization methods.

Proposed penalties effectively balance excess risks in regularization.

Applicable to severely ill-posed high-dimensional problems.

Abstract

This paper deals with recovering an unknown vector $\theta$ from the noisy data $Y=A\theta+\sigma\xi$, where $A$ is a known $(m\times n)$-matrix and $\xi$ is a white Gaussian noise. It is assumed that $n$ is large and $A$ may be severely ill-posed. Therefore, in order to estimate $\theta$, a spectral regularization method is used, and our goal is to choose its regularization parameter with the help of the data $Y$. For spectral regularization methods related to the so-called ordered smoothers [see Kneip Ann. Statist. 22 (1994) 835--866], we propose new penalties in the principle of empirical risk minimization. The heuristical idea behind these penalties is related to balancing excess risks. Based on this approach, we derive a sharp oracle inequality controlling the mean square risks of data-driven spectral regularization methods.