Text Indexing and Searching in Sublinear Time

TL;DR

This paper presents a novel text index that can be built and queried in sublinear time, significantly improving efficiency for pattern matching in large texts, with extensions to secondary memory models.

Contribution

Introduces the first sublinear-time index for text building and querying, utilizing a novel difference cover sampling technique for efficient longest common prefix computations.

Findings

Index can be built in o(n) time and queried in o(q) time

Supports pattern occurrence counting and locating efficiently

Extends to secondary memory with near-optimal I/O performance

Abstract

We introduce the first index that can be built in $o(n)$ time for a text of length $n$, and can also be queried in $o(q)$ time for a pattern of length $q$. On an alphabet of size $\sigma$, our index uses $O(n\sqrt{\log n\log\sigma})$ bits, is built in $O(n((\log\log n)^2+\sqrt{\log\sigma})/\sqrt{\log_\sigma n})$ deterministic time, and computes the number $\mathrm{occ}$ of occurrences of the pattern in time $O(q/\log_\sigma n+\log n)$. Each such occurrence can then be found in $O(\sqrt{\log n\log\sigma})$ time. By slightly increasing the space and construction time, to $O(n(\sqrt{\log n\log\sigma}+ \log\sigma\log^\varepsilon n))$ and $O(n\log^{3/2}\sigma/\log^{1/2-\varepsilon} n)$, respectively, for any constant $0<\varepsilon<1/2$, we can find the $\mathrm{occ}$ pattern occurrences in time $O(q/\log_\sigma n + \sqrt{\log_\sigma n}\log\log n + \mathrm{occ})$. We build on a novel text sampling based on difference covers, which enjoys properties that allow us efficiently computing longest common prefixes in constant time. We extend our results to the secondary memory model as well, where we give the first construction in $o(\mathit{Sort}(n))$ I/Os of a data structure with suffix array functionality; this data structure supports pattern matching queries with optimal or nearly-optimal cost.