DSA: Scalable Distributed Sequence Alignment System Using SIMD Instructions

TL;DR

DSA is a scalable distributed sequence alignment system that combines Spark and SIMD instructions to significantly accelerate bioinformatics sequence alignment tasks in distributed environments.

Contribution

The paper introduces DSA, a novel system that integrates Spark with SIMD-based data parallelism for efficient, scalable sequence alignment.

Findings

Achieves up to 201x speedup over SparkSW.

Exhibits near linear scalability with increasing cluster nodes.

Demonstrates outstanding performance in distributed sequence alignment.

Abstract

Sequence alignment algorithms are a basic and critical component of many bioinformatics fields. With rapid development of sequencing technology, the fast growing reference database volumes and longer length of query sequence become new challenges for sequence alignment. However, the algorithm is prohibitively high in terms of time and space complexity. In this paper, we present DSA, a scalable distributed sequence alignment system that employs Spark to process sequences data in a horizontally scalable distributed environment, and leverages data parallel strategy based on Single Instruction Multiple Data (SIMD) instruction to parallelize the algorithm in each core of worker node. The experimental results demonstrate that 1) DSA has outstanding performance and achieves up to 201x speedup over SparkSW. 2) DSA has excellent scalability and achieves near linear speedup when increasing the number of nodes in cluster.