Entropy coding with Variable Length Re-writing Systems

TL;DR

This paper introduces a new class of variable length re-writing codes for data compression that can outperform Huffman and Hu-Tucker codes in mean description length and offer additional properties like lexicographic order preservation and balanced bit probability.

Contribution

It proposes novel code construction methods within variable length re-writing systems, enhancing compression efficiency and encoding properties over traditional VLCs.

Findings

Codes can have shorter mean description length than Huffman codes.

Lexicographic order can be preserved in the bit domain.

Codes can achieve balanced bit probability without knowing the source distribution.

Abstract

This paper describes a new set of block source codes well suited for data compression. These codes are defined by sets of productions rules of the form a.l->b, where a in A represents a value from the source alphabet A and l, b are -small- sequences of bits. These codes naturally encompass other Variable Length Codes (VLCs) such as Huffman codes. It is shown that these codes may have a similar or even a shorter mean description length than Huffman codes for the same encoding and decoding complexity. A first code design method allowing to preserve the lexicographic order in the bit domain is described. The corresponding codes have the same mean description length (mdl) as Huffman codes from which they are constructed. Therefore, they outperform from a compression point of view the Hu-Tucker codes designed to offer the lexicographic property in the bit domain. A second construction method allows to obtain codes such that the marginal bit probability converges to 0.5 as the sequence length increases and this is achieved even if the probability distribution function is not known by the encoder.