TetRex: a novel algorithm for index-accelerated search of highly conserved motifs

TL;DR

TetRex is a new algorithm that improves the speed of searching for conserved biological sequences using a novel data structure.

Contribution

The novel contribution is a regex search algorithm that leverages hierarchical interleaved bloom filters to reduce search runtime.

Findings

TetRex reduces the search space for regex queries in biological datasets.

The algorithm outperforms state-of-the-art tools when combined with traditional search methods.

Abstract

The scale of modern datasets has necessitated innovations to solve even the most traditional and fundamental of computational problems. Set membership and set cardinality are both examples of simple queries that, for large enough datasets, quickly become challenging using traditional approaches. Interestingly, we find a need for these innovations within the field of biology. Despite the vast differences among living organisms, there exist functions so critical to life that they are conserved in the genomes and proteomes across seemingly unrelated species. Regular expressions (regexes) can serve as a convenient way to represent these conserved sequences compactly. However, despite the strong theoretical foundation and maturity of tools available, the state-of-the-art regex search falls short of what is necessary to quickly scan an enormous database of biological sequences. In this work, we describe a novel algorithm for regex search that reduces the search space by leveraging a recently developed compact data structure for set membership, the hierarchical interleaved bloom filter. We show that the runtime of our method combined with a traditional search outperforms state-of-the-art tools.

Figures