CLaDMoP: Learning Transferrable Models from Successful Clinical Trials via LLMs

TL;DR

CLaDMoP introduces a novel pre-training approach using large language models and a multi-level fusion technique to improve clinical trial outcome prediction, achieving significant performance gains over existing methods.

Contribution

The paper presents CLaDMoP, a new pre-training method that leverages LLMs and a pair matching proxy task, enhancing generalization and performance in clinical trial outcome prediction.

Findings

Significantly improves PR-AUC and ROC-AUC over baselines.

Achieves up to 10.5% improvement in PR-AUC.

Performs well after Parameter-Efficient Fine-Tuning.

Abstract

Many existing models for clinical trial outcome prediction are optimized using task-specific loss functions on trial phase-specific data. While this scheme may boost prediction for common diseases and drugs, it can hinder learning of generalizable representations, leading to more false positives/negatives. To address this limitation, we introduce CLaDMoP, a new pre-training approach for clinical trial outcome prediction, alongside the Successful Clinical Trials dataset(SCT), specifically designed for this task. CLaDMoP leverages a Large Language Model-to encode trials' eligibility criteria-linked to a lightweight Drug-Molecule branch through a novel multi-level fusion technique. To efficiently fuse long embeddings across levels, we incorporate a grouping block, drastically reducing computational overhead. CLaDMoP avoids reliance on task-specific objectives by pre-training on a "pair matching" proxy task. Compared to established zero-shot and few-shot baselines, our method significantly improves both PR-AUC and ROC-AUC, especially for phase I and phase II trials. We further evaluate and perform ablation on CLaDMoP after Parameter-Efficient Fine-Tuning, comparing it to state-of-the-art supervised baselines, including MEXA-CTP, on the Trial Outcome Prediction(TOP) benchmark. CLaDMoP achieves up to 10.5% improvement in PR-AUC and 3.6% in ROC-AUC, while attaining comparable F1 score to MEXA-CTP, highlighting its potential for clinical trial outcome prediction. Code and SCT dataset can be downloaded from https://github.com/murai-lab/CLaDMoP.