Benchmarking drug-drug interaction prediction methods: a perspective of distribution changes

TL;DR

This paper introduces DDI-Ben, a benchmarking framework that simulates distribution changes in drug-drug interaction prediction, revealing current methods' vulnerabilities and highlighting the robustness of LLM-based approaches.

Contribution

The paper presents DDI-Ben, a novel benchmarking framework that simulates real-world distribution changes for emerging DDI prediction, and provides extensive evaluation of existing methods.

Findings

Most existing methods degrade significantly under distribution changes.

LLM-based methods and textual information integration show improved robustness.

The benchmark datasets with simulated distribution changes are publicly released.

Abstract

Motivation: Emerging drug-drug interaction (DDI) prediction is crucial for new drugs but is hindered by distribution changes between known and new drugs in real-world scenarios. Current evaluation often neglects these changes, relying on unrealistic i.i.d. split due to the absence of drug approval data. Results: We propose DDI-Ben, a benchmarking framework for emerging DDI prediction under distribution changes. DDI-Ben introduces a distribution change simulation framework that leverages distribution changes between drug sets as a surrogate for real-world distribution changes of DDIs, and is compatible with various drug split strategies. Through extensive benchmarking on ten representative methods, we show that most existing approaches suffer substantial performance degradation under distribution changes. Our analysis further indicates that large language model (LLM) based methods and the integration of drug-related textual information offer promising robustness against such degradation. To support future research, we release the benchmark datasets with simulated distribution changes. Overall, DDI-Ben highlights the importance of explicitly addressing distribution changes and provides a foundation for developing more resilient methods for emerging DDI prediction. Availability and implementation: Our code and data are available at https://github.com/LARS-research/DDI-Bench.