About adaptive coding on countable alphabets

TL;DR

This paper analyzes universal coding for countable alphabets with specific hazard rate conditions, proving the adaptiveness of the AC code and providing bounds based on metric entropy and sample maxima.

Contribution

It demonstrates the adaptiveness of the auto-censuring AC code for classes of sources with finite non-decreasing hazard rates, extending understanding of universal coding.

Findings

AC code is adaptive for specified source classes

Provides non-asymptotic bounds for maxima of discrete samples

Characterizes universal redundancy rate via metric entropy

Abstract

This paper sheds light on universal coding with respect to classes of memoryless sources over a countable alphabet defined by an envelope function with finite and non-decreasing hazard rate. We prove that the auto-censuring AC code introduced by Bontemps (2011) is adaptive with respect to the collection of such classes. The analysis builds on the tight characterization of universal redundancy rate in terms of metric entropy % of small source classes by Opper and Haussler (1997) and on a careful analysis of the performance of the AC-coding algorithm. The latter relies on non-asymptotic bounds for maxima of samples from discrete distributions with finite and non-decreasing hazard rate.