Partial Data Compression and Text Indexing via Optimal Suffix Multi-Selection

TL;DR

This paper introduces an optimal adaptive method for partial suffix-based data compression and text indexing, significantly reducing computation time for extracting segments from large texts by solving the suffix multi-selection problem efficiently.

Contribution

It presents a novel optimal algorithm for suffix multi-selection, improving partial suffix computation times to O(K log K + N), matching bounds for atomic element selection.

Findings

Partial suffix computations achieved in O(K log K + N) time.

Suffix multi-selection solved optimally with Theta(N log N - sum Delta_j log Delta_j + N).

Method applies to data compression and text indexing applications.

Abstract

Consider an input text string T[1,N] drawn from an unbounded alphabet. We study partial computation in suffix-based problems for Data Compression and Text Indexing such as (I) retrieve any segment of K<=N consecutive symbols from the Burrows-Wheeler transform of T, and (II) retrieve any chunk of K<=N consecutive entries of the Suffix Array or the Suffix Tree. Prior literature would take O(N log N) comparisons (and time) to solve these problems by solving the total problem of building the entire Burrows-Wheeler transform or Text Index for T, and performing a post-processing to single out the wanted portion. We introduce a novel adaptive approach to partial computational problems above, and solve both the partial problems in O(K log K + N) comparisons and time, improving the best known running times of O(N log N) for K=o(N). These partial-computation problems are intimately related since they share a common bottleneck: the suffix multi-selection problem, which is to output the suffixes of rank r_1,r_2,...,r_K under the lexicographic order, where r_1<r_2<...<r_K, r_i in [1,N]. Special cases of this problem are well known: K=N is the suffix sorting problem that is the workhorse in Stringology with hundreds of applications, and K=1 is the recently studied suffix selection. We show that suffix multi-selection can be solved in Theta(N log N - sum_{j=0}^K Delta_j log Delta_j+N) time and comparisons, where r_0=0, r_{K+1}=N+1, and Delta_j=r_{j+1}-r_j for 0<=j<=K. This is asymptotically optimal, and also matches the bound in [Dobkin, Munro, JACM 28(3)] for multi-selection on atomic elements (not suffixes). Matching the bound known for atomic elements for strings is a long running theme and challenge from 70's, which we achieve for the suffix multi-selection problem. The partial suffix problems as well as the suffix multi-selection problem have many applications.