TrinityDNA: A Bio-Inspired Foundational Model for Efficient Long-Sequence DNA Modeling

TL;DR

TrinityDNA is a biologically inspired foundational model that effectively captures long-range dependencies in DNA sequences, improving genomic analysis tasks through innovative structural components and training strategies.

Contribution

It introduces TrinityDNA, a novel DNA model with biologically informed modules and a multi-scale attention mechanism, advancing long-sequence DNA modeling capabilities.

Findings

Enhanced gene function prediction accuracy

Improved regulatory mechanism discovery

Established a new DNA long-sequence CDS annotation benchmark

Abstract

The modeling of genomic sequences presents unique challenges due to their length and structural complexity. Traditional sequence models struggle to capture long-range dependencies and biological features inherent in DNA. In this work, we propose TrinityDNA, a novel DNA foundational model designed to address these challenges. The model integrates biologically informed components, including Groove Fusion for capturing DNA's structural features and Gated Reverse Complement (GRC) to handle the inherent symmetry of DNA sequences. Additionally, we introduce a multi-scale attention mechanism that allows the model to attend to varying levels of sequence dependencies, and an evolutionary training strategy that progressively adapts the model to both prokaryotic and eukaryotic genomes. TrinityDNA provides a more accurate and efficient approach to genomic sequence modeling, offering significant improvements in gene function prediction, regulatory mechanism discovery, and other genomics applications. Our model bridges the gap between machine learning techniques and biological insights, paving the way for more effective analysis of genomic data. Additionally, we introduced a new DNA long-sequence CDS annotation benchmark to make evaluations more comprehensive and oriented toward practical applications.