Concentration in unbounded metric spaces and algorithmic stability

TL;DR

This paper extends McDiarmid's inequality to unbounded metric spaces using a new subgaussian diameter concept, enabling dimension-free concentration results and a novel generalization bound for unbounded loss functions in algorithmic stability.

Contribution

It introduces the subgaussian diameter for unbounded metric spaces and applies it to derive new concentration inequalities and generalization bounds.

Findings

Extended McDiarmid's inequality for unbounded spaces

Introduced the subgaussian diameter concept

Provided the first generalization bound for unbounded loss functions

Abstract

We prove an extension of McDiarmid's inequality for metric spaces with unbounded diameter. To this end, we introduce the notion of the {\em subgaussian diameter}, which is a distribution-dependent refinement of the metric diameter. Our technique provides an alternative approach to that of Kutin and Niyogi's method of weakly difference-bounded functions, and yields nontrivial, dimension-free results in some interesting cases where the former does not. As an application, we give apparently the first generalization bound in the algorithmic stability setting that holds for unbounded loss functions. We furthermore extend our concentration inequality to strongly mixing processes.