Universally Typical Sets for Ergodic Sources of Multidimensional Data

TL;DR

This paper extends the theory of universally typical sets and empirical entropy estimation from one-dimensional to multidimensional ergodic sources, using packing and covering techniques to construct sample sets.

Contribution

It introduces a multidimensional framework for universally typical sets and empirical entropy estimation, generalizing existing one-dimensional results.

Findings

Constructs sequences of multidimensional array sets with probability one

Sets' cardinality grows at most exponentially with rate h_0

Extends ergodic source sampling theory to multidimensional data

Abstract

We lift important results about universally typical sets, typically sampled sets, and empirical entropy estimation in the theory of samplings of discrete ergodic information sources from the usual one-dimensional discrete-time setting to a multidimensional lattice setting. We use techniques of packings and coverings with multidimensional windows to construct sequences of multidimensional array sets which in the limit build the generated samples of any ergodic source of entropy rate below an $h_0$ with probability one and whose cardinality grows at most at exponential rate $h_0$.