Redundancy Estimates for Word-Based Encoding of Sequences Produced by a Bernoulli Source

TL;DR

This paper introduces word-based encoding schemes for Bernoulli sequences that achieve low redundancy comparable to block codes but with significantly reduced complexity, and discusses their synchronization properties.

Contribution

It constructs word-based codes with redundancy decreasing as the negative five-thirds power of average delay, improving efficiency and complexity over traditional methods.

Findings

Word-based codes can match block-code redundancy with lower complexity.

Uniform output codes avoid synchronization errors.

Redundancy scales as ar N^{-5/3} for the proposed codes.

Abstract

The efficiency of a code is estimated by its redundancy $R$, while the complexity of a code is estimated by its average delay $\bar N$. In this work we construct word-based codes, for which $R \lesssim \bar N^{-5/3}$. Therefore, word-based codes can attain the same redundancy as block-codes while being much less complex. We also consider uniform on the output codes, the benefit of which is the lack of a running synchronization error. For such codes $\bar N^{-1} \lesssim R \lesssim \bar N^{-1}$, except for a case when all input symbols are equiprobable, when $R \leqslant \bar N^{-2}$ for infinitely many $\bar N$.