Learning subgaussian classes : Upper and minimax bounds

TL;DR

This paper derives sharp bounds for empirical risk minimization in subgaussian regression models, demonstrating optimal error rates and providing new proofs for minimax results in Gaussian settings.

Contribution

It provides the first sharp oracle inequalities for ERM in subgaussian classes and extends minimax results with new proofs for Gaussian regression.

Findings

Sharp oracle inequalities for ERM in subgaussian models

ERM achieves optimal error rates under mild conditions

New proof techniques for minimax results in Gaussian regression

Abstract

We obtain sharp oracle inequalities for the empirical risk minimization procedure in the regression model under the assumption that the target Y and the model F are subgaussian. The bound we obtain is sharp in the minimax sense if F is convex. Moreover, under mild assumptions on F, the error rate of ERM remains optimal even if the procedure is allowed to perform with constant probability. A part of our analysis is a new proof of minimax results for the gaussian regression model.