CrossLLM-Mamba: Multimodal State Space Fusion of LLMs for RNA Interaction Prediction

TL;DR

CrossLLM-Mamba introduces a dynamic, state-space approach to RNA interaction prediction using multimodal LLMs, significantly improving accuracy and robustness over static fusion methods.

Contribution

It presents a novel state-space alignment framework with bidirectional encoders for dynamic, context-aware multimodal interaction modeling in biological data.

Findings

Achieves MCC of 0.892 on RPI1460 benchmark, surpassing previous best by 5.2%.

Attains Pearson correlation >0.95 in binding affinity predictions for certain RNA subtypes.

Maintains linear complexity, enabling scalable high-dimensional multimodal modeling.

Abstract

Accurate prediction of RNA-associated interactions is essential for understanding cellular regulation and advancing drug discovery. While Biological Large Language Models (BioLLMs) such as ESM-2 and RiNALMo provide powerful sequence representations, existing methods rely on static fusion strategies that fail to capture the dynamic, context-dependent nature of molecular binding. We introduce CrossLLM-Mamba, a novel framework that reformulates interaction prediction as a state-space alignment problem. By leveraging bidirectional Mamba encoders, our approach enables deep ``crosstalk'' between modality-specific embeddings through hidden state propagation, modeling interactions as dynamic sequence transitions rather than static feature overlaps. The framework maintains linear computational complexity, making it scalable to high-dimensional BioLLM embeddings. We further incorporate Gaussian noise injection and Focal Loss to enhance robustness against hard-negative samples. Comprehensive experiments across three interaction categories, RNA-protein, RNA-small molecule, and RNA-RNA demonstrate that CrossLLM-Mamba achieves state-of-the-art performance. On the RPI1460 benchmark, our model attains an MCC of 0.892, surpassing the previous best by 5.2\%. For binding affinity prediction, we achieve Pearson correlations exceeding 0.95 on riboswitch and repeat RNA subtypes. These results establish state-space modeling as a powerful paradigm for multi-modal biological interaction prediction.