Generalization bounds for nonparametric regression with $\beta-$mixing samples

TL;DR

This paper extends uniform deviation inequalities and generalization bounds from independent to dependent $eta$-mixing samples in nonparametric regression, broadening theoretical analysis to dependent data scenarios.

Contribution

It provides a direct method to adapt deviation inequalities and generalization bounds from independent to $eta$-mixing dependent samples in nonparametric regression.

Findings

Derived uniform deviation inequalities for $eta$-mixing samples.

Extended VC and similar theories to dependent data.

Applicable to geometrically ergodic Markov chains.

Abstract

In this paper we present a series of results that permit to extend in a direct manner uniform deviation inequalities of the empirical process from the independent to the dependent case characterizing the additional error in terms of $\beta-$mixing coefficients associated to the training sample. We then apply these results to some previously obtained inequalities for independent samples associated to the deviation of the least-squared error in nonparametric regression to derive corresponding generalization bounds for regression schemes in which the training sample may not be independent. These results provide a framework to analyze the error associated to regression schemes whose training sample comes from a large class of $\beta-$mixing sequences, including geometrically ergodic Markov samples, using only the independent case. More generally, they permit a meaningful extension of the Vapnik-Chervonenkis and similar theories for independent training samples to this class of $\beta-$mixing samples.