MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language

TL;DR

MAMMAL is a multi-modal foundation model that integrates diverse biological data types to improve prediction and generation tasks in drug discovery, achieving state-of-the-art results across multiple benchmarks.

Contribution

The paper introduces MAMMAL, a versatile multi-task model that unifies biological data modalities for enhanced predictive capabilities, surpassing prior task-specific models.

Findings

Achieves SOTA in 9 out of 11 downstream tasks.

Demonstrates improved classification in antibody-antigen binding prediction.

Supports diverse biological data modalities within a single architecture.

Abstract

Large language models applied to vast biological datasets have the potential to transform biology by uncovering disease mechanisms and accelerating drug development. However, current models are often siloed, trained separately on small-molecules, proteins, or transcriptomic data, limiting their ability to capture complex, multi-modal interactions. Effective drug discovery requires computational tools that integrate multiple biological entities while supporting prediction and generation, a challenge existing models struggle to address. For this purpose, we present MAMMAL - Molecular Aligned Multi-Modal Architecture and Language - a versatile method applied to create a multi-task foundation model that learns from large-scale biological datasets across diverse modalities, including proteins, small-molecules, and omics. MAMMAL's structured prompt syntax supports classification, regression, and generation tasks while handling token and scalar inputs and outputs. Evaluated on eleven diverse downstream tasks, it reaches a new state of the art (SOTA) in nine tasks and is comparable to SOTA in two tasks, all within a unified architecture, unlike prior task-specific models. Additionally, we explored Alphafold 3 binding prediction capabilities on antibody-antigen and nanobody-antigen complexes showing significantly better classification performance of MAMMAL in 3 out of 4 targets. The model code and pretrained weights are publicly available at https://github.com/BiomedSciAI/biomed-multi-alignment and https://huggingface.co/ibm/biomed.omics.bl.sm.ma-ted-458m