Basic inequalities for weighted entropies

TL;DR

This paper develops fundamental inequalities for weighted entropies, extending classical entropy bounds by incorporating outcome-dependent weights, with applications to matrix inequalities and information measures.

Contribution

It introduces new inequalities for weighted entropies, generalizing classical bounds and providing a framework for weighted information measures.

Findings

Weighted Ky Fan and Hadamard inequalities involving determinants.

Weighted Cramér-Rao inequalities with weighted Fisher information.

Extensions of standard entropy bounds to weighted entropy context.

Abstract

The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. In this paper, we establish a number of simple inequalities for the weighted entropies (general as well as specific), mirroring similar bounds on standard (Shannon) entropies and related quantities. The required assumptions are written in terms of various expectations of the weight functions. Examples are weighted Ky Fan and weighted Hadamard inequalities involving determinants of positive-definite matrices, and weighted Cram\'{e}r-Rao inequalities involving the weighted Fisher information matrix.