scDrugMap: Benchmarking Large Foundation Models for Drug Response Prediction

TL;DR

scDrugMap is a comprehensive benchmarking framework that evaluates large foundation models for predicting drug response in single-cell data, demonstrating superior performance and facilitating drug discovery research.

Contribution

It introduces the first large-scale benchmark of foundation models for drug response prediction in single-cell data, including a versatile platform for evaluation and comparison.

Findings

scFoundation achieved the highest performance in pooled-data evaluation.

UCE excelled in cross-data evaluation after fine-tuning.

scGPT led in zero-shot learning scenarios.

Abstract

Drug resistance presents a major challenge in cancer therapy. Single cell profiling offers insights into cellular heterogeneity, yet the application of large-scale foundation models for predicting drug response in single cell data remains underexplored. To address this, we developed scDrugMap, an integrated framework featuring both a Python command-line interface and a web server for drug response prediction. scDrugMap evaluates a wide range of foundation models, including eight single-cell models and two large language models, using a curated dataset of over 326,000 cells in the primary collection and 18,800 cells in the validation set, spanning 36 datasets and diverse tissue and cancer types. We benchmarked model performance under pooled-data and cross-data evaluation settings, employing both layer freezing and Low-Rank Adaptation (LoRA) fine-tuning strategies. In the pooled-data scenario, scFoundation achieved the best performance, with mean F1 scores of 0.971 (layer freezing) and 0.947 (fine-tuning), outperforming the lowest-performing model by over 50%. In the cross-data setting, UCE excelled post fine-tuning (mean F1: 0.774), while scGPT led in zero-shot learning (mean F1: 0.858). Overall, scDrugMap provides the first large-scale benchmark of foundation models for drug response prediction in single-cell data and serves as a user-friendly, flexible platform for advancing drug discovery and translational research.