dnaHNet: A Scalable and Hierarchical Foundation Model for Genomic Sequence Learning

TL;DR

dnaHNet is a scalable, tokenizer-free genomic model that adaptively compresses DNA sequences, enabling efficient, interpretable predictions and hierarchical biological insights without supervision.

Contribution

It introduces a novel differentiable chunking mechanism for end-to-end genomic modeling, outperforming existing architectures in efficiency and scalability.

Findings

Outperforms leading models in scaling and efficiency.

Achieves over 3x inference speedup compared to Transformers.

Automatically discovers hierarchical biological structures.

Abstract

Genomic foundation models have the potential to decode DNA syntax, yet face a fundamental tradeoff in their input representation. Standard fixed-vocabulary tokenizers fragment biologically meaningful motifs such as codons and regulatory elements, while nucleotide-level models preserve biological coherence but incur prohibitive computational costs for long contexts. We introduce dnaHNet, a state-of-the-art tokenizer-free autoregressive model that segments and models genomic sequences end-to-end. Using a differentiable dynamic chunking mechanism, dnaHNet compresses raw nucleotides into latent tokens adaptively, balancing compression with predictive accuracy. Pretrained on prokaryotic genomes, dnaHNet outperforms leading architectures including StripedHyena2 in scaling and efficiency. This recursive chunking yields quadratic FLOP reductions, enabling $>3 \times$ inference speedup over Transformers. On zero-shot tasks, dnaHNet achieves superior performance in predicting protein variant fitness and gene essentiality, while automatically discovering hierarchical biological structures without supervision. These results establish dnaHNet as a scalable, interpretable framework for next-generation genomic modeling.