BioBridge: Bridging Biomedical Foundation Models via Knowledge Graphs

TL;DR

BioBridge is a novel framework that uses knowledge graphs to connect unimodal biomedical foundation models, enabling effective multimodal tasks without retraining the original models.

Contribution

It introduces a parameter-efficient method to bridge unimodal models using knowledge graphs, facilitating multimodal biomedical applications without fine-tuning base models.

Findings

Outperforms baseline KG embedding methods by 76.3% in cross-modal retrieval.

Demonstrates strong out-of-domain generalization to unseen modalities.

Enhances biomedical multimodal question answering and drug generation.

Abstract

Foundation models (FMs) are able to leverage large volumes of unlabeled data to demonstrate superior performance across a wide range of tasks. However, FMs developed for biomedical domains have largely remained unimodal, i.e., independently trained and used for tasks on protein sequences alone, small molecule structures alone, or clinical data alone. To overcome this limitation of biomedical FMs, we present BioBridge, a novel parameter-efficient learning framework, to bridge independently trained unimodal FMs to establish multimodal behavior. BioBridge achieves it by utilizing Knowledge Graphs (KG) to learn transformations between one unimodal FM and another without fine-tuning any underlying unimodal FMs. Our empirical results demonstrate that BioBridge can beat the best baseline KG embedding methods (on average by around 76.3%) in cross-modal retrieval tasks. We also identify BioBridge demonstrates out-of-domain generalization ability by extrapolating to unseen modalities or relations. Additionally, we also show that BioBridge presents itself as a general purpose retriever that can aid biomedical multimodal question answering as well as enhance the guided generation of novel drugs.