Fixed-to-Variable Length Resolution Coding for Target Distributions

TL;DR

This paper introduces a fixed-to-variable length coding scheme that efficiently approximates target distributions with minimal divergence, outperforming traditional block-to-block methods especially at finite lengths.

Contribution

A novel fixed-to-variable length encoder using M-type quantization and Tunstall coding is developed, achieving zero divergence asymptotically and superior finite-length performance.

Findings

Encoder achieves zero un-normalized informational divergence asymptotically.

Significantly outperforms optimal block-to-block encoders at finite lengths.

Number of random bits per symbol equals the entropy of the target distribution.

Abstract

The number of random bits required to approximate a target distribution in terms of un-normalized informational divergence is considered. It is shown that for a variable-to-variable length encoder, this number is lower bounded by the entropy of the target distribution. A fixed-to-variable length encoder is constructed using M-type quantization and Tunstall coding. It is shown that the encoder achieves in the limit an un-normalized informational divergence of zero with the number of random bits per generated symbol equal to the entropy of the target distribution. Numerical results show that the proposed encoder significantly outperforms the optimal block-to-block encoder in the finite length regime.