RLBWT Tricks

TL;DR

This paper refines a recent constant-time backward-stepping method for run-length compressed BWT, demonstrating practical improvements and tradeoffs through experiments on genomic data.

Contribution

It introduces a refined approach with a time-space tradeoff, a permutation decomposition scheme for better compression, and experimental validation on real datasets.

Findings

Backward-stepping is faster without compression but uses more space.

Compressed data structures achieve a good balance of time and space.

Practical implementation shows near-constant query times with compression.

Abstract

Until recently, most experts would probably have agreed we cannot backwards-step in constant time with a run-length compressed Burrows-Wheeler Transform (RLBWT), since doing so relies on rank queries on sparse bitvectors and those inherit lower bounds from predecessor queries. At ICALP '21, however, Nishimoto and Tabei described a new, simple and constant-time implementation. For a permutation $\pi$, it stores an $O (r)$-space table -- where $r$ is the number of positions $i$ where either $i = 0$ or $\pi (i + 1) \neq \pi (i) + 1$ -- that enables the computation of successive values of $\pi(i)$ by table look-ups and linear scans. Nishimoto and Tabei showed how to increase the number of rows in the table to bound the length of the linear scans such that the query time for computing $\pi(i)$ is constant while maintaining $O (r)$-space. In this paper we refine Nishimoto and Tabei's approach, including a time-space tradeoff, and experimentally evaluate different implementations demonstrating the practicality of part of their result. We show that even without adding rows to the table, in practice we almost always scan only a few entries during queries. We propose a decomposition scheme of the permutation $\pi$ corresponding to the LF-mapping that allows an improved compression of the data structure, while limiting the query time. We tested our implementation on real-world genomic datasets and found that without compression of the table, backward-stepping is drastically faster than with sparse bitvector implementations but, unfortunately, also uses drastically more space. After compression, backward-stepping is competitive both in time and space with the best existing implementations.