Variable-Length Source Dispersions Differ under Maximum and Average Error Criteria

TL;DR

This paper investigates variable-length source coding with side-information, allowing non-zero error probability, and reveals that source dispersion differs under maximum and average error criteria, with implications for guessing problems.

Contribution

The paper introduces new one-shot bounds using cutoff entropies and derives second-order asymptotics, highlighting the difference in source dispersion between error criteria.

Findings

Source dispersion under average error is generally smaller than under maximum error.

First-order asymptotics are identical for both error criteria.

Applications to guessing problems demonstrate practical relevance.

Abstract

Variable-length compression without prefix-free constraints and with side-information available at both encoder and decoder is considered. Instead of requiring the code to be error-free, we allow for it to have a non-vanishing error probability. We derive one-shot bounds on the optimal average codeword length by proposing two new information quantities; namely, the conditional and unconditional $\varepsilon$-cutoff entropies. Using these one-shot bounds, we obtain the second-order asymptotics of the problem under two different formalisms---the average and maximum probabilities of error over the realization of the side-information. While the first-order terms in the asymptotic expansions for both formalisms are identical, we find that the source dispersion under the average error formalism is, in most cases, strictly smaller than its maximum error counterpart. Applications to a certain class of guessing problems, previously studied by Kuzuoka [\emph{{IEEE} Trans.\ Inf.\ Theory}, vol.~66, no.~3, pp.~1674--1690, 2020], are also discussed.