Burrows-Wheeler transform for terabases

TL;DR

This paper presents a scalable algorithm for constructing the Burrows-Wheeler transform of massive genomic datasets, enabling efficient reference-free genome analysis at terabase scales.

Contribution

It introduces a practical method to merge BWTs of subcollections, facilitating handling of terabases of sequencing data efficiently.

Findings

Merges BWTs of large read collections efficiently

Processes 600 Gbp/day on a single system

Uses 30 GB memory overhead

Abstract

In order to avoid the reference bias introduced by mapping reads to a reference genome, bioinformaticians are investigating reference-free methods for analyzing sequenced genomes. With large projects sequencing thousands of individuals, this raises the need for tools capable of handling terabases of sequence data. A key method is the Burrows-Wheeler transform (BWT), which is widely used for compressing and indexing reads. We propose a practical algorithm for building the BWT of a large read collection by merging the BWTs of subcollections. With our 2.4 Tbp datasets, the algorithm can merge 600 Gbp/day on a single system, using 30 gigabytes of memory overhead on top of the run-length encoded BWTs.