Fano's inequality for random variables

TL;DR

This paper generalizes Fano's inequality to apply to arbitrary events and random variables, providing new tools for analyzing Bayesian posterior rates and sequential learning regret.

Contribution

It extends Fano's inequality to non-partitioned events and continuous random variables, enabling new proofs in Bayesian and sequential learning contexts.

Findings

New bounds for Bayesian posterior concentration rates

Improved regret bounds in non-stochastic sequential learning

Generalized Fano's inequality applicable to arbitrary events and variables

Abstract

We extend Fano's inequality, which controls the average probability of events in terms of the average of some $f$--divergences, to work with arbitrary events (not necessarily forming a partition) and even with arbitrary $[0,1]$--valued random variables, possibly in continuously infinite number. We provide two applications of these extensions, in which the consideration of random variables is particularly handy: we offer new and elegant proofs for existing lower bounds, on Bayesian posterior concentration (minimax or distribution-dependent) rates and on the regret in non-stochastic sequential learning.