PyTrial: Machine Learning Software and Benchmark for Clinical Trial Applications

TL;DR

PyTrial is an open-source framework that benchmarks and provides tools for applying machine learning algorithms to various clinical trial tasks, facilitating research and development in this domain.

Contribution

It introduces a comprehensive benchmark suite with datasets, implementations, and a modular API for ML in clinical trials, addressing data accessibility and task standardization issues.

Findings

34 ML algorithms evaluated across 6 clinical trial tasks

23 datasets prepared with working examples in Jupyter Notebooks

Framework simplifies implementation with a four-step process

Abstract

Clinical trials are conducted to test the effectiveness and safety of potential drugs in humans for regulatory approval. Machine learning (ML) has recently emerged as a new tool to assist in clinical trials. Despite this progress, there have been few efforts to document and benchmark ML4Trial algorithms available to the ML research community. Additionally, the accessibility to clinical trial-related datasets is limited, and there is a lack of well-defined clinical tasks to facilitate the development of new algorithms. To fill this gap, we have developed PyTrial that provides benchmarks and open-source implementations of a series of ML algorithms for clinical trial design and operations. In this paper, we thoroughly investigate 34 ML algorithms for clinical trials across 6 different tasks, including patient outcome prediction, trial site selection, trial outcome prediction, patient-trial matching, trial similarity search, and synthetic data generation. We have also collected and prepared 23 ML-ready datasets as well as their working examples in Jupyter Notebooks for quick implementation and testing. PyTrial defines each task through a simple four-step process: data loading, model specification, model training, and model evaluation, all achievable with just a few lines of code. Furthermore, our modular API architecture empowers practitioners to expand the framework to incorporate new algorithms and tasks effortlessly. The code is available at https://github.com/RyanWangZf/PyTrial.