Third-Order Asymptotics of Variable-Length Compression Allowing Errors

TL;DR

This paper derives the third-order asymptotics for variable-length source coding allowing errors without prefix-free constraints, revealing the third-order term's dependence on the error probability.

Contribution

It provides the first detailed third-order asymptotic analysis for non-prefix-free variable-length compression with errors, linking fixed-length and variable-length limits.

Findings

Third-order asymptotics depend on error probability

Refined analysis uses moderate and strong large deviations

Results differ from traditional prefix-free coding asymptotics

Abstract

This study investigates the fundamental limits of variable-length compression in which prefix-free constraints are not imposed (i.e., one-to-one codes are studied) and non-vanishing error probabilities are permitted. Due in part to a crucial relation between the variable-length and fixed-length compression problems, our analysis requires a careful and refined analysis of the fundamental limits of fixed-length compression in the setting where the error probabilities are allowed to approach either zero or one polynomially in the blocklength. To obtain the refinements, we employ tools from moderate deviations and strong large deviations. Finally, we provide the third-order asymptotics for the problem of variable-length compression with non-vanishing error probabilities. We show that unlike several other information-theoretic problems in which the third-order asymptotics are known, for the problem of interest here, the third-order term depends on the permissible error probability.