BioLangFusion: Multimodal Fusion of DNA, mRNA, and Protein Language Models

TL;DR

BioLangFusion introduces a biologically aligned multimodal fusion method for DNA, mRNA, and protein language models, enhancing molecular property prediction without additional pre-training.

Contribution

It presents a novel biologically motivated alignment and fusion approach for integrating pre-trained molecular language models at the codon level.

Findings

Outperforms unimodal baselines on five molecular property tasks.

Simple fusion methods effectively capture complementary multi-omic information.

No extra pre-training required for integration.

Abstract

We present BioLangFusion, a simple approach for integrating pre-trained DNA, mRNA, and protein language models into unified molecular representations. Motivated by the central dogma of molecular biology (information flow from gene to transcript to protein), we align per-modality embeddings at the biologically meaningful codon level (three nucleotides encoding one amino acid) to ensure direct cross-modal correspondence. BioLangFusion studies three standard fusion techniques: (i) codon-level embedding concatenation, (ii) entropy-regularized attention pooling inspired by multiple-instance learning, and (iii) cross-modal multi-head attention -- each technique providing a different inductive bias for combining modality-specific signals. These methods require no additional pre-training or modification of the base models, allowing straightforward integration with existing sequence-based foundation models. Across five molecular property prediction tasks, BioLangFusion outperforms strong unimodal baselines, showing that even simple fusion of pre-trained models can capture complementary multi-omic information with minimal overhead.