Probabilistic analysis of arithmetic coding showing its robustness

TL;DR

This paper provides a probabilistic analysis of arithmetic coding, demonstrating its robustness across different bit distributions and showing that the final interval converges to a uniform distribution regardless of the source bias.

Contribution

It introduces a functional equation for the moment generating function and proves the convergence of the final interval to a uniform distribution for all p, highlighting robustness.

Findings

Final interval converges to uniform distribution on [0,1]

Robustness of arithmetic coding across all p values

Rate of convergence varies with p

Abstract

We probabilistically analyze the performance of the arithmetic coding algorithm under a probability model for binary data in which a message is received by a coder from a source emitting independent equally distributed bits, with 1 occurring with probability $p\in(0,1)$ and 0 occurring with probability $1-p$. We establish a functional equation for the bivariate moment generating function for the two ends of the final interval delivered by the algorithm. Via the method of moments, we show that the transmitted message converges in distribution to the standard continuous uniform random variable on the interval [0,1]. It is remarkable that the limiting distribution is the same for all $p$, indicating robustness in the performance of arithmetic coding across an entire family of bit distributions. The nuance with $p$ appears only in the rate of convergence.