Pattern Coding Meets Censoring: (almost) Adaptive Coding on Countable Alphabets

TL;DR

This paper introduces an adaptive coding scheme for sources over infinite alphabets characterized by envelope classes, achieving near-minimax redundancy with a simpler, efficient online prefix code that covers a broad class of distributions.

Contribution

It presents a novel adaptive coding method for infinite alphabet sources with regular variation envelopes, improving redundancy and simplicity over prior approaches.

Findings

Achieves near-minimax redundancy within a $ ext{log} ext{log} n$ factor.

Constructs a computationally efficient online prefix code.

Covers the full range of slowly varying envelope classes.

Abstract

Adaptive coding faces the following problem: given a collection of source classes such that each class in the collection has non-trivial minimax redundancy rate, can we design a single code which is asymptotically minimax over each class in the collection? In particular, adaptive coding makes sense when there is no universal code on the union of classes in the collection. In this paper, we deal with classes of sources over an infinite alphabet, that are characterized by a dominating envelope. We provide asymptotic equivalents for the redundancy of envelope classes enjoying a regular variation property. We finally construct a computationally efficient online prefix code, which interleaves the encoding of the so-called pattern of the message and the encoding of the dictionary of discovered symbols. This code is shown to be adaptive, within a $\log\log n$ factor, over the collection of regularly varying envelope classes. The code is both simpler and less redundant than previously described contenders. In contrast with previous attempts, it also covers the full range of slowly varying envelope classes.