BarcodeMamba: State Space Models for Biodiversity Analysis

TL;DR

BarcodeMamba introduces a state space model-based foundation for DNA barcode analysis, outperforming previous models like BarcodeBERT in species identification accuracy with fewer parameters, especially in recognizing unseen species.

Contribution

We developed BarcodeMamba, a novel state space model for DNA barcodes, demonstrating superior performance and efficiency over existing models like BarcodeBERT in biodiversity analysis.

Findings

BarcodeMamba achieves 99.2% accuracy on species-level identification for seen species.

It attains 70.2% genus-level accuracy for unseen species with fewer parameters.

BarcodeMamba outperforms BarcodeBERT in both accuracy and parameter efficiency.

Abstract

DNA barcodes are crucial in biodiversity analysis for building automatic identification systems that recognize known species and discover unseen species. Unlike human genome modeling, barcode-based invertebrate identification poses challenges in the vast diversity of species and taxonomic complexity. Among Transformer-based foundation models, BarcodeBERT excelled in species-level identification of invertebrates, highlighting the effectiveness of self-supervised pretraining on barcode-specific datasets. Recently, structured state space models (SSMs) have emerged, with a time complexity that scales sub-quadratically with the context length. SSMs provide an efficient parameterization of sequence modeling relative to attention-based architectures. Given the success of Mamba and Mamba-2 in natural language, we designed BarcodeMamba, a performant and efficient foundation model for DNA barcodes in biodiversity analysis. We conducted a comprehensive ablation study on the impacts of self-supervised training and tokenization methods, and compared both versions of Mamba layers in terms of expressiveness and their capacity to identify "unseen" species held back from training. Our study shows that BarcodeMamba has better performance than BarcodeBERT even when using only 8.3% as many parameters, and improves accuracy to 99.2% on species-level accuracy in linear probing without fine-tuning for "seen" species. In our scaling study, BarcodeMamba with 63.6% of BarcodeBERT's parameters achieved 70.2% genus-level accuracy in 1-nearest neighbor (1-NN) probing for unseen species. The code repository to reproduce our experiments is available at https://github.com/bioscan-ml/BarcodeMamba.