Lower-bounds on the Bayesian Risk in Estimation Procedures via $f$-Divergences

TL;DR

This paper develops a general lower-bound on Bayesian estimation risk using $f$-divergences, improving existing bounds based on mutual information by incorporating alternative divergence measures like Maximal Leakage and Hellinger.

Contribution

It introduces a unified lower-bound framework for Bayesian estimation risk that extends beyond mutual information, encompassing a broader class of $f$-divergences.

Findings

The new bounds outperform mutual information-based bounds in certain settings.

Application of the bounds to specific estimation problems demonstrates their effectiveness.

Comparison shows the generalized bounds can be tighter than existing results.

Abstract

We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of $f$-Divergences. The results are then applied to specific settings of interest and compared to other notable results in the literature. In particular, we show that the known bounds using Mutual Information can be improved by using, for example, Maximal Leakage, Hellinger divergence, or generalizations of the Hockey-Stick divergence.