Universal Coding on Infinite Alphabets: Exponentially Decreasing Envelopes

TL;DR

This paper investigates universal lossless coding for infinite alphabets with exponentially decreasing envelope classes, establishing redundancy bounds, proposing a coding strategy, and providing an adaptive algorithm with optimal redundancy performance.

Contribution

It introduces a new analysis of minimax redundancy for exponentially decreasing envelope classes and develops an adaptive coding algorithm achieving this redundancy.

Findings

Minimax redundancy is proportional to (1/4α log e) log^2 n.

Proposed coding strategy matches Bayes redundancy with maximin redundancy.

Adaptive algorithm attains the minimax redundancy asymptotically.

Abstract

This paper deals with the problem of universal lossless coding on a countable infinite alphabet. It focuses on some classes of sources defined by an envelope condition on the marginal distribution, namely exponentially decreasing envelope classes with exponent $\alpha$. The minimax redundancy of exponentially decreasing envelope classes is proved to be equivalent to $\frac{1}{4 \alpha \log e} \log^2 n$. Then a coding strategy is proposed, with a Bayes redundancy equivalent to the maximin redundancy. At last, an adaptive algorithm is provided, whose redundancy is equivalent to the minimax redundancy