Adaptive encodings for small and fast compressed suffix arrays

TL;DR

This paper introduces variable-length blocking (VLB), an adaptive encoding method for compressed suffix arrays that balances space and query speed by tailoring auxiliary data to local compressibility, outperforming existing methods.

Contribution

The paper proposes VLB, a novel encoding technique that adaptively compresses BWT regions, improving space efficiency and query performance in compressed suffix arrays.

Findings

VLB outperforms the r-index and sr-index in query time.

VLB retains space close to the sr-index.

VLB offers a better space-time tradeoff than the move data structure.

Abstract

Compressed suffix arrays (CSAs) index large repetitive collections and are key in many text applications. The r-index and its derivatives combine the run-length Burrows-Wheeler Transform (BWT) with suffix array sampling to achieve space proportional to the number of equal-symbol runs in the BWT. While effective for near-identical strings, their size grows quickly as variation increases, since the number of BWT runs is sensitive to edits. Existing approaches typically trade space for query speed, or vice versa, limiting their practicality at large scale. We introduce variable-length blocking (VLB), an encoding technique for BWT-based CSAs that adapts the amount of indexing information to local compressibility. The BWT is recursively divided into blocks of at most w runs (a parameter) and organized into a tree. Compressible regions appear near the root and store little auxiliary data, while incompressible regions lie deeper and retain additional information to speed up access. Queries traverse a short root-to-leaf path followed by a small run scan. This strategy balances space and query speed by transferring bits saved in compressible areas to accelerate access in incompressible ones. Backward search relies on rank and successor queries over the BWT. We introduce a sampling technique that guarantees correctness only along valid backward-search states, reducing space without affecting query performance. We extend VLB to encode the subsampled r-index (sr-index). Experiments show that VLB-based techniques outperform the r-index and sr-index in query time, while retaining space close to that of the sr-index. Compared to the move data structure, VLB offers a more favorable space-time tradeoff.