Hierarchical Molecular Language Models (HMLMs)

TL;DR

HMLMs are a transformer-based framework modeling cellular signaling as a molecular language, integrating multi-scale data to predict dynamics and uncover pathway interactions, advancing biology-oriented AI applications.

Contribution

The paper introduces HMLMs, a novel hierarchical transformer architecture that models cellular signaling as a molecular language, integrating multi-modal data and capturing biological hierarchies.

Findings

HMLMs outperform traditional models in temporal dynamics prediction.

Attention analysis reveals meaningful crosstalk patterns.

Framework enables biology-oriented large language models for signaling data.

Abstract

Artificial intelligence (AI) is reshaping computational and network biology by enabling new approaches to decode cellular communication networks. We introduce Hierarchical Molecular Language Models (HMLMs), a novel framework that models cellular signaling as a specialized molecular language, where signaling molecules function as tokens, protein interactions define syntax, and functional consequences constitute semantics. HMLMs employ a transformer-based architecture adapted to accommodate graph-structured signaling networks through information transducers, mathematical entities that capture how molecules receive, process, and transmit signals. The architecture integrates multi-modal data sources across molecular, pathway, and cellular scales through hierarchical attention mechanisms and scale-bridging operators that enable information flow across biological hierarchies. Applied to a complex network of cardiac fibroblast signaling, HMLMs outperformed traditional approaches in temporal dynamics prediction, particularly under sparse sampling conditions. Attention-based analysis revealed biologically meaningful crosstalk patterns, including previously uncharacterized interactions between signaling pathways. By bridging molecular mechanisms with cellular phenotypes through AI-driven molecular language representation, HMLMs establish a foundation for biology-oriented large language models (LLMs) that could be pre-trained on comprehensive pathway datasets and applied across diverse signaling systems and tissues, advancing precision medicine and therapeutic discovery.