Generalization Bounds for Markov Algorithms through Entropy Flow Computations

TL;DR

This paper extends entropy flow methods to derive generalization bounds for all Markov process-based learning algorithms, connecting their ergodic properties to generalization error.

Contribution

It introduces a new exact entropy flow formula for Markov algorithms and links it to modified logarithmic Sobolev inequalities, broadening the applicability of existing techniques.

Findings

Derived new generalization bounds for several algorithms

Extended entropy flow analysis to all Markov process-based algorithms

Connected ergodic properties of Markov processes to generalization error

Abstract

Many learning algorithms can be represented as Markov processes, and understanding their generalization error is a central topic in learning theory. For specific continuous-time noisy algorithms, a prominent analysis technique relies on information-theoretic tools and the so-called ``entropy flow'' method. This technique is compatible with a broad range of assumptions and leverages the convergence properties of learning dynamics to produce meaningful generalization bounds, which can also be informative or extend to discrete-time settings. Despite their success, existing entropy flow formulations are limited to specific noise and algorithm structures (\eg, Langevin dynamics). In this work, we exploit new technical tools to extend its applicability to all learning algorithms whose iterative dynamics is governed by a time-homogeneous Markov process. Our approach builds on a principled continuous-time approximation of Markov algorithms and introduces a new, exact entropy flow formula for such processes. Within this unified framework, we establish novel connections to a well-studied family of modified logarithmic Sobolev inequalities, which we use to connect the generalization error to the ergodic properties of Markov processes. Finally, we provide a detailed analysis of all the terms appearing in our theory and demonstrate its effectiveness by deriving new generalization bounds for several concrete algorithms.