Kolmogorov--Nagumo Mean Frameworks for Conditional Entropy

TL;DR

This paper introduces new Kolmogorov--Nagumo mean-based frameworks for conditional entropy, extending existing models and capturing entropies beyond traditional representations, with implications for information theory properties.

Contribution

It proposes a novel generalized framework for $g$-conditional entropies that extends beyond existing $( ext{eta},F)$-entropy models and satisfies key information-theoretic properties.

Findings

Introduces the $( ext{eta}, ext{psi})$-KN averaging framework and proves its equivalence to existing models.

Shows existence of certain entropies not representable by traditional $( ext{eta},F)$-entropies but captured by the new framework.

Derives conditions under which the new entropies satisfy conditioning reduces entropy and data-processing inequality.

Abstract

This study focuses on conditional entropy frameworks based on the Kolmogorov--Nagumo (KN) mean. First, $(\eta, \psi)$-KN averaging (\texttt{EPKNAVG}), a KN-mean extension of the $\eta$-averaging (\texttt{EAVG}) framework for $(\eta, F)$-entropies, is introduced and proven to be equivalent to \texttt{EAVG} under suitable concavification conditions. Second, motivated by generalized $g$-vulnerability, a new framework is proposed for generalized $g$-conditional entropies. This framework captures conditional entropies beyond the scope of \texttt{EAVG}-type representations. In particular, it is shown that there exists an $\alpha$ and a joint probability distribution $p_{X, Y}$ such that the Augustin--Csisz{\' a}r conditional entropy $H_{\alpha}^{\mathrm{C}}(X|Y)$ cannot be represented by any $(\eta,F)$-entropy satisfying \texttt{EAVG}. In contrast, it is represented within the proposed framework. Furthermore, sufficient conditions are derived under which the proposed generalized $g$-conditional entropies satisfy the conditioning reduces entropy property and the data-processing inequality.