Generalization bounds for learning under graph-dependence: A survey

TL;DR

This survey reviews recent advances in understanding how dependence among data points, modeled by graphs, affects generalization bounds in statistical learning, highlighting new concentration bounds and their applications.

Contribution

It compiles and discusses graph-dependent concentration bounds and their use in deriving generalization bounds, providing a comprehensive overview of this emerging research area.

Findings

Collected various graph-dependent concentration bounds

Derived Rademacher complexity and stability bounds for dependent data

Outlined practical learning scenarios and future research directions

Abstract

Traditional statistical learning theory relies on the assumption that data are identically and independently distributed (i.i.d.). However, this assumption often does not hold in many real-life applications. In this survey, we explore learning scenarios where examples are dependent and their dependence relationship is described by a dependency graph, a commonly utilized model in probability and combinatorics. We collect various graph-dependent concentration bounds, which are then used to derive Rademacher complexity and stability generalization bounds for learning from graph-dependent data. We illustrate this paradigm through practical learning tasks and provide some research directions for future work. To our knowledge, this survey is the first of this kind on this subject.