Optimum Overflow Thresholds in Variable-Length Source Coding Allowing Non-Vanishing Error Probability

TL;DR

This paper investigates the optimal overflow thresholds in variable-length source coding with non-zero error probability, providing bounds, formulas, and analysis for both general and specific source models.

Contribution

It derives finite-length bounds and formulas for the optimum overflow thresholds, extending understanding beyond mean codeword length to overflow probabilities.

Findings

Derived finite-length upper and lower bounds on overflow probabilities

Established formulas for optimum overflow thresholds in first- and second-order forms

Applied formulas to stationary memoryless sources

Abstract

The variable-length source coding problem allowing the error probability up to some constant is considered for general sources. In this problem the optimum mean codeword length of variable-length codes has already been determined. On the other hand, in this paper, we focus on the overflow (or excess codeword length) probability instead of the mean codeword length. The infimum of overflow thresholds under the constraint that both of the error probability and the overflow probability are smaller than or equal to some constant is called the optimum overflow threshold. In this paper, we first derive finite-length upper and lower bounds on these probabilities so as to analyze the optimum overflow thresholds. Then, by using these bounds we determine the general formula of the optimum overflow thresholds in both of the first-order and second-order forms. Next, we consider another expression of the derived general formula so as to reveal the relationship with the optimum coding rate in the fixed-length source coding problem. We also derive general formulas of the optimum overflow thresholds in the optimistic coding scenario. Finally, we apply general formulas derived in this paper to the case of stationary memoryless sources.