Source coding with escort distributions and Renyi entropy bounds

TL;DR

This paper explores the use of escort distributions and Renyi entropy in source coding, introducing new length measures and demonstrating their bounds and optimality properties.

Contribution

It introduces a new family of length measures based on escort distributions and establishes their bounds via Renyi entropy, linking standard and escort distributions.

Findings

Optimal codes can be derived using escort-distributions.

Generalized length measures are bounded by Renyi entropy.

Shannon codes are optimal for the new length measures.

Abstract

We discuss the interest of escort distributions and R\'enyi entropy in the context of source coding. We first recall a source coding theorem by Campbell relating a generalized measure of length to the R\'enyi-Tsallis entropy. We show that the associated optimal codes can be obtained using considerations on escort-distributions. We propose a new family of measure of length involving escort-distributions and we show that these generalized lengths are also bounded below by the R\'enyi entropy. Furthermore, we obtain that the standard Shannon codes lengths are optimum for the new generalized lengths measures, whatever the entropic index. Finally, we show that there exists in this setting an interplay between standard and escort distributions.