A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise

TL;DR

This paper introduces a new concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise, applicable to a broad class of models.

Contribution

It extends existing results by providing a general inequality for quadratic contrasts, beyond linear cases, under minimal model assumptions.

Findings

Established a new concentration inequality for excess risk

Applied the inequality to quadratic contrasts in regression

Enhanced understanding of empirical process behavior in heteroscedastic models

Abstract

We prove a new and general concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise. No specific structure is required on the model, except the existence of a suitable function that controls the local suprema of the empirical process. So far, only the case of linear contrast estimation was tackled in the literature with this level of generality on the model. We solve here the case of a quadratic contrast, by separating the behavior of a linearized empirical process and the empirical process driven by the squares of functions of models.