A Bernstein-type inequality for suprema of random processes with applications to model selection in non-Gaussian regression

TL;DR

This paper develops Bernstein-type exponential bounds for the supremum of certain random processes using generic chaining, and applies these bounds to improve model selection methods in non-Gaussian regression with independent, Laplace-transform-admitting errors.

Contribution

It introduces weaker-condition exponential bounds for suprema of random processes and applies them to enhance model selection in non-Gaussian regression.

Findings

Established Bernstein-type inequalities for suprema of random processes.

Applied bounds to develop a model selection procedure with oracle inequalities.

Demonstrated the effectiveness of the approach in non-Gaussian regression settings.

Abstract

Let $\pa{X_{t}}_{t\in T}$ be a family of real-valued centered random variables indexed by a countable set $T$. In the first part of this paper, we establish exponential bounds for the deviation probabilities of the supremum $Z=\sup_{t\in T}X_{t}$ by using the generic chaining device introduced in Talagrand (2005). Compared to concentration-type inequalities, these bounds offer the advantage to hold under weaker conditions on the family $\pa{X_{t}}_{t\in T}$. The second part of the paper is oriented towards statistics. We consider the regression setting $Y=f+\eps$ where $f$ is an unknown vector of $\R^{n}$ and $\eps$ is a random vector the components of which are independent, centered and admit finite Laplace transforms in a neighborhood of 0. Our aim is to estimate $f$ from the observation of $Y$ by mean of a model selection approach among a collection of linear subspaces of $\R^{n}$. The selection procedure we propose is based on the minimization of a penalized criterion the penalty of which is calibrated by using the deviation bounds established in the first part of this paper. More precisely, we study suprema of random variables of the form $X_{t}=\sum_{i=1}^{n}t_{i}\eps_{i}$ when $t$ varies among the unit ball of a linear subspace of $\R^{n}$. We finally show that our estimator satisfies some oracle-type inequality under suitable assumptions on the metric structures of the linear spaces of the collection.