A Bernstein-type inequality for suprema of random processes with an application to statistics

TL;DR

This paper develops exponential deviation bounds for suprema of random processes using Talagrand's generic chaining, and applies these results to statistical model selection with non-Gaussian errors.

Contribution

It introduces a Bernstein-type inequality for suprema of random processes and applies it to improve model selection techniques in regression with non-Gaussian errors.

Findings

Established exponential bounds on supremum deviations.

Derived deviation inequalities for Euclidean norms of projected random vectors.

Applied inequalities to model selection in regression with large model collections.

Abstract

We use the generic chaining device proposed by Talagrand to establish exponential bounds on the deviation probability of some suprema of random processes. Then, given a random vector $\xi$ in $\R^{n}$ the components of which are independent and admit a suitable exponential moment, we deduce a deviation inequality for the squared Euclidean norm of the projection of $\xi$ onto a linear subspace of $\R^{n}$. Finally, we provide an application of such an inequality to statistics, performing model selection in the regression setting when the errors are possibly non-Gaussian and the collection of models possibly large.