CTP-LLM: Clinical Trial Phase Transition Prediction Using Large Language Models

TL;DR

This paper introduces CTP-LLM, a novel GPT-3.5-based model that predicts clinical trial phase transitions from protocol texts, achieving over 67% accuracy and demonstrating the potential of LLMs in clinical trial outcome forecasting.

Contribution

It presents the first LLM-based approach for clinical trial outcome prediction and introduces the PhaseTransition dataset as a benchmark for this task.

Findings

67% overall accuracy in phase transition prediction

75% accuracy for Phase III to approval transition

Demonstrates LLMs' potential in clinical trial analysis

Abstract

New medical treatment development requires multiple phases of clinical trials. Despite the significant human and financial costs of bringing a drug to market, less than 20% of drugs in testing will make it from the first phase to final approval. Recent literature indicates that the design of the trial protocols significantly contributes to trial performance. We investigated Clinical Trial Outcome Prediction (CTOP) using trial design documents to predict phase transitions automatically. We propose CTP-LLM, the first Large Language Model (LLM) based model for CTOP. We also introduce the PhaseTransition (PT) Dataset; which labels trials based on their progression through the regulatory process and serves as a benchmark for CTOP evaluation. Our fine-tuned GPT-3.5-based model (CTP-LLM) predicts clinical trial phase transition by analyzing the trial's original protocol texts without requiring human-selected features. CTP-LLM achieves a 67% accuracy rate in predicting trial phase transitions across all phases and a 75% accuracy rate specifically in predicting the transition from Phase~III to final approval. Our experimental performance highlights the potential of LLM-powered applications in forecasting clinical trial outcomes and assessing trial design.