Weighted entropy: basic inequalities

TL;DR

This paper extends the concept of weighted entropy, analyzing its fundamental inequalities and connections with Fisher information and entropy power, providing a deeper understanding of context-dependent information measures.

Contribution

It introduces and analyzes inequalities for weighted entropy, including Fisher information and entropy power inequalities, and explores their theoretical connections.

Findings

Weighted entropy inequalities analogous to classical ones are established.

Connections between weighted entropy and Fisher information are analyzed.

The concepts of weighted entropy rates are discussed.

Abstract

This paper represents an extended version of an earlier note [10]. The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. We analyse analogs of the Fisher information inequality and entropy power inequality for the weighted entropy and discuss connections with weighted Lieb's splitting inequality. The concepts of rates of the weighted entropy and information are also discussed.