LLMs for Drug-Drug Interaction Prediction: A Comprehensive Comparison

TL;DR

This study explores the use of Large Language Models for predicting drug-drug interactions, demonstrating that fine-tuned LLMs outperform traditional methods and can effectively process molecular and biological data.

Contribution

It is the first comprehensive evaluation of LLMs' capabilities in DDI prediction, including fine-tuning and validation across multiple datasets, showing their potential in pharmaceutical research.

Findings

Fine-tuned LLMs achieved high sensitivity and accuracy in DDI prediction.

Phi-3.5 2.7B model reached 97.8% sensitivity and 91.9% accuracy.

LLMs outperform traditional machine learning approaches in DDI prediction.

Abstract

The increasing volume of drug combinations in modern therapeutic regimens needs reliable methods for predicting drug-drug interactions (DDIs). While Large Language Models (LLMs) have revolutionized various domains, their potential in pharmaceutical research, particularly in DDI prediction, remains largely unexplored. This study thoroughly investigates LLMs' capabilities in predicting DDIs by uniquely processing molecular structures (SMILES), target organisms, and gene interaction data as raw text input from the latest DrugBank dataset. We evaluated 18 different LLMs, including proprietary models (GPT-4, Claude, Gemini) and open-source variants (from 1.5B to 72B parameters), first assessing their zero-shot capabilities in DDI prediction. We then fine-tuned selected models (GPT-4, Phi-3.5 2.7B, Qwen-2.5 3B, Gemma-2 9B, and Deepseek R1 distilled Qwen 1.5B) to optimize their performance. Our comprehensive evaluation framework included validation across 13 external DDI datasets, comparing against traditional approaches such as l2-regularized logistic regression. Fine-tuned LLMs demonstrated superior performance, with Phi-3.5 2.7B achieving a sensitivity of 0.978 in DDI prediction, with an accuracy of 0.919 on balanced datasets (50% positive, 50% negative cases). This result represents an improvement over both zero-shot predictions and state-of-the-art machine-learning methods used for DDI prediction. Our analysis reveals that LLMs can effectively capture complex molecular interaction patterns and cases where drug pairs target common genes, making them valuable tools for practical applications in pharmaceutical research and clinical settings.