Parallel String Graph Construction and Transitive Reduction for De Novo Genome Assembly

TL;DR

This paper introduces distributed-memory parallel algorithms for overlap detection and transitive reduction in de novo genome assembly, significantly improving efficiency and scalability for large genomes using long reads.

Contribution

The authors develop novel distributed algorithms based on linear algebra for overlap detection and layout simplification, implemented in the diBELLA 2D pipeline, advancing large genome assembly.

Findings

Achieves near linear scaling with over 80% efficiency on the human genome.

Reduces runtime for overlap detection by 1.2-1.3x for human genome.

Transitive reduction outperforms existing methods by up to 29x.

Abstract

One of the most computationally intensive tasks in computational biology is de novo genome assembly, the decoding of the sequence of an unknown genome from redundant and erroneous short sequences. A common assembly paradigm identifies overlapping sequences, simplifies their layout, and creates consensus. Despite many algorithms developed in the literature, the efficient assembly of large genomes is still an open problem. In this work, we introduce new distributed-memory parallel algorithms for overlap detection and layout simplification steps of de novo genome assembly, and implement them in the diBELLA 2D pipeline. Our distributed memory algorithms for both overlap detection and layout simplification are based on linear-algebra operations over semirings using 2D distributed sparse matrices. Our layout step consists of performing a transitive reduction from the overlap graph to a string graph. We provide a detailed communication analysis of the main stages of our new algorithms. diBELLA 2D achieves near linear scaling with over 80% parallel efficiency for the human genome, reducing the runtime for overlap detection by 1.2-1.3x for the human genome and 1.5-1.9x for C. elegans compared to the state-of-the-art. Our transitive reduction algorithm outperforms an existing distributed-memory implementation by 10.5-13.3x for the human genome and 18-29x for the C. elegans. Our work paves the way for efficient de novo assembly of large genomes using long reads in distributed memory.