Faster Compact On-Line Lempel-Ziv Factorization

TL;DR

This paper introduces a faster on-line algorithm for Lempel-Ziv factorization that improves time complexity to O(N log N) using DAWGs, and offers an efficient variant for run-length compressed strings.

Contribution

The paper presents a novel on-line Lempel-Ziv factorization algorithm utilizing Directed Acyclic Word Graphs, achieving significant improvements in time and space efficiency over prior methods.

Findings

Achieves O(N log N) time complexity with low space usage.

Introduces an opportunistic variant for run-length compressed strings.

Demonstrates faster and more space-efficient factorization for compressible data.

Abstract

We present a new on-line algorithm for computing the Lempel-Ziv factorization of a string that runs in $O(N\log N)$ time and uses only $O(N\log\sigma)$ bits of working space, where $N$ is the length of the string and $\sigma$ is the size of the alphabet. This is a notable improvement compared to the performance of previous on-line algorithms using the same order of working space but running in either $O(N\log^3N)$ time (Okanohara & Sadakane 2009) or $O(N\log^2N)$ time (Starikovskaya 2012). The key to our new algorithm is in the utilization of an elegant but less popular index structure called Directed Acyclic Word Graphs, or DAWGs (Blumer et al. 1985). We also present an opportunistic variant of our algorithm, which, given the run length encoding of size $m$ of a string of length $N$, computes the Lempel-Ziv factorization on-line, in $O\left(m \cdot \min \left\{\frac{(\log\log m)(\log \log N)}{\log\log\log N}, \sqrt{\frac{\log m}{\log \log m}} \right\}\right)$ time and $O(m\log N)$ bits of space, which is faster and more space efficient when the string is run-length compressible.