On how generalised entropies without parameters impact information optimisation processes

TL;DR

This paper explores a family of non-parametric, non-extensive entropies for information theory, demonstrating their effectiveness in data compression and channel capacity in low-density regions, while aligning with Shannon's theory at high densities.

Contribution

It introduces a new family of generalized entropies that are asymptotically equivalent to Shannon's but differ in low-density regions, with proven coding theorems and applications to channel capacity.

Findings

Improved data compression in low-density regions.

Enhanced channel capacity maximization using generalized entropies.

Results align with Shannon's theory at high densities.

Abstract

As an application of generalised statistical mechanics, it is studied a possible route toward a consistent generalised information theory in terms of a family of non-extensive, non-parametric entropies $H^\pm_D(P)$. Unlike other proposals based on non-extensive entropies with a parameter dependence, our scheme is asymptotically equivalent to the one formulated by Shannon, while it differs in regions where the density of states is reasonably small, which leads to information distributions constrained to their background. Two basic concepts are discussed to this aim. First, we prove two effective coding theorems for the entropies $H^\pm_D(P)$. Then we calculate the channel capacity of a binary symmetric channel (BSC) and a binary erasure channel (BEC) in terms of these entropies. We found that processes such as data compression and channel capacity maximisation can be improved in regions where there is a low density of states, whereas for high densities our results coincide with Shannon's formulation.