Online-to-PAC generalization bounds under graph-mixing dependencies

TL;DR

This paper develops new generalization bounds for learning algorithms under graph-mixing dependencies, combining temporal and graph-based dependency models to quantify how data dependencies affect learning guarantees.

Contribution

It introduces a framework that models dependencies decaying with graph distance and derives generalization bounds using an online-to-PAC approach, integrating graph structure into the analysis.

Findings

Bounds depend on mixing rate and graph chromatic number

Provides concentration results for dependent data

Bridges temporal and graph dependency models

Abstract

Traditional generalization results in statistical learning require a training data set made of independently drawn examples. Most of the recent efforts to relax this independence assumption have considered either purely temporal (mixing) dependencies, or graph-dependencies, where non-adjacent vertices correspond to independent random variables. Both approaches have their own limitations, the former requiring a temporal ordered structure, and the latter lacking a way to quantify the strength of inter-dependencies. In this work, we bridge these two lines of work by proposing a framework where dependencies decay with graph distance. We derive generalization bounds leveraging the online-to-PAC framework, by deriving a concentration result and introducing an online learning framework incorporating the graph structure. The resulting high-probability generalization guarantees depend on both the mixing rate and the graph's chromatic number.