# BioFuse: an embedding fusion framework for biomedical foundation models

## TL;DR

BioFuse is a new framework that combines biomedical foundation models to improve performance and uncover cross-modal relationships in data.

## Contribution

BioFuse introduces a novel embedding fusion framework using multiple pre-trained models and grid search for optimal combinations.

## Key findings

- BioFuse achieves state-of-the-art AUC in 5/12 datasets on the MedMNIST+ benchmark.
- The framework demonstrates unexpected cross-modal capabilities across imaging modalities.
- BioFuse offers an extensible architecture for future model integration and fusion techniques.

## Abstract

The biomedical field has witnessed a surge in pre-trained foundation models excelling in specific subdomains such as radiology and histopathology. While integrating these models promises a more comprehensive understanding of biomedical data, it poses challenges in model compatibility and feature fusion. We present BioFuse, a novel open-source framework designed to generate optimised biomedical embeddings. BioFuse utilises a pool of 9 state-of-the-art (SOTA) foundation models to create task-specific embeddings. It employs grid search to automatically identify the optimal combination of models, fusing their embeddings through vector concatenation. On the MedMNIST+ benchmark, using XGBoost as the downstream classifier, BioFuse outperforms several existing methods, achieving SOTA AUC in 5/12 datasets, while maintaining near-SOTA performance across most remaining datasets. Remarkably, our experiments reveal unexpected cross-modal capabilities, with histopathology and radiology models showing strong performance when applied to other imaging modalities. BioFuse features a high-level API for immediate deployment and an extensible architecture to incorporate future models and fusion techniques. We anticipate BioFuse will not only enhance the utility of foundation models in biomedicine but also open new avenues for uncovering cross-modal relationships in biomedical data.