Towards Better Compressed Representations

TL;DR

This paper presents a heuristic for computing parsings with bounded phrase length that minimize zeroth order entropy, improving succinct text representations and providing bounds based on empirical entropy measures.

Contribution

It introduces a novel heuristic for bounded-phrase-length parsing that minimizes entropy and offers practical bounds related to empirical entropy measures.

Findings

Heuristic improves text compression by minimizing entropy.

Structured parsing bounds relate to empirical entropy measures.

Practical application in succinct text representations.

Abstract

We introduce the problem of computing a parsing where each phrase is of length at most $m$ and which minimizes the zeroth order entropy of parsing. Based on the recent theoretical results we devise a heuristic for this problem. The solution has straightforward application in succinct text representations and gives practical improvements. Moreover the proposed heuristic yields structure whose size can be bounded both by $|S|H_{m-1}(S)$ and by $|S|/m(H_0(S) + \cdots + H_{m-1})$, where $H_{k}(S)$ is the $k$-th order empirical entropy of $S$. We also consider a similar problem in which the first-order entropy is minimized.