Dynamic Grammar-Compressed Self-Index in $\delta$-Optimal Space

TL;DR

This paper introduces the first dynamic self-index that is both space-efficient for highly repetitive data and supports fast updates and queries, advancing the state of the art in self-indexing.

Contribution

It presents the dynamic RR-index, achieving delta-optimal space and efficient query and update times, overcoming limitations of previous dynamic self-indexes.

Findings

Occupies expected delta-optimal space in practice.

Answers locate queries efficiently with no dependence on LCP.

Supports insertions and deletions with expected amortized time.

Abstract

A compressed self-index stores a string in compressed form while supporting locate queries without decompression. For highly repetitive strings (arising in web crawls, versioned documents, and genomic collections), static self-indexes can match the $\delta$-optimal lower bound of $\Omega(\delta \log(n \log \sigma / (\delta \log n)) \log n)$ bits up to constant factors, where $n$ is the string length, $\sigma$ is the alphabet size, and $\delta$ is the substring complexity. Their dynamic counterparts, however, remain scarce: every existing dynamic self-index either fails to attain $\delta$-optimal space, pays at least $\Theta(\log n)$ time per reported occurrence during locate, or exposes the longest common prefix (LCP) of the text inside its update time. We present the dynamic RR-index, a dynamic grammar-compressed self-index built on the restricted recompression run-length straight-line program (RLSLP). To our knowledge, it is the first dynamic self-index to attain $\delta$-optimal space. The index occupies expected $O(\delta \log(n \log \sigma / (\delta \log n)) \log n)$ bits, answers locate queries in expected $O(m + \log m \log^{2} n + \mathit{occ} (\log n / \log \log n))$ time (where $m$ is the pattern length and $\mathit{occ}$ is the number of occurrences), and supports insertions and deletions of a length-$m'$ substring in expected amortized $O(m' \log^{2} n + \log^{3} n)$ time, with no dependence on the LCP. On eleven highly repetitive corpora, including a $37$ GB Wikipedia dump and a $59$ GB human-chromosome collection, the dynamic RR-index is up to $77\times$ faster than the dynamic r-index on updates and up to $11\times$ faster than other dynamic indexes on locate.