Entropy bounds for grammar compression

TL;DR

This paper establishes bounds on the size of grammar compression encodings, explaining practical efficiency and limitations of methods like RePair and Greedy, and introduces new entropy bounds for string parsing.

Contribution

It provides theoretical bounds for common grammar compression encodings, explaining practical performance and limitations of RePair and Greedy algorithms, and introduces new entropy bounds for string parsing.

Findings

RePair's standard encoding achieves 1.5|S|H_k(S) size.

Stopping after certain iterations achieves |S|H_k(S) size.

The analysis explains why some methods outperform others in practice.

Abstract

Grammar compression represents a string as a context free grammar. Achieving compression requires encoding such grammar as a binary string; there are a few commonly used encodings. We bound the size of practically used encodings for several heuristical compression methods, including \RePair and \Greedy algorithms: the standard encoding of \RePair, which combines entropy coding and special encoding of a grammar, achieves $1.5|S|H_k(S)$, where $H_k(S)$ is $k$-th order entropy of $S$. We also show that by stopping after some iteration we can achieve $|S|H_k(S)$. This is particularly interesting, as it explains a phenomenon observed in practice: introducing too many nonterminals causes the bit-size to grow. We generalize our approach to other compression methods like \Greedy and a wide class of irreducible grammars as well as to other practically used bit encodings (including naive, which uses fixed-length codes). Our approach not only proves the bounds but also partially explains why \Greedy and \RePair are much better in practice than other grammar based methods. In some cases we argue that our estimations are optimal. The tools used in our analysis are of independent interest: we prove the new, optimal, bounds on the zeroth order entropy of parsing of a string.