Evaluating the Effectiveness of Personalized Medicine with Software

TL;DR

This paper introduces an open-source software tool for evaluating personalized medicine models, enabling practitioners to assess their effectiveness compared to standard care using bootstrap inference on clinical trial data.

Contribution

The paper presents a novel R package, PTE, that facilitates out-of-sample evaluation of personalized treatment models, including binary and survival endpoints, with confidence intervals and hypothesis testing.

Findings

Software successfully evaluates treatment improvement in simulated data.

Application to depression trial data demonstrates practical utility.

Method provides statistically rigorous assessment of personalization benefits.

Abstract

We present methodological advances in understanding the effectiveness of personalized medicine models and supply easy-to-use open-source software. Personalized medicine involves the systematic use of individual patient characteristics to determine which treatment option is most likely to result in a better outcome for the patient on average. Why is personalized medicine not done more in practice? One of many reasons is because practitioners do not have any easy way to holistically evaluate whether their personalization procedure does better than the standard of care. Our software, "Personalized Treatment Evaluator" (the R package PTE), provides inference for improvement out-of-sample in many clinical scenarios. We also extend current methodology by allowing evaluation of improvement in the case where the endpoint is binary or survival. In the software, the practitioner inputs (1) data from a single-stage randomized trial with one continuous, incidence or survival endpoint and (2) a functional form of a model for the endpoint constructed from domain knowledge. The bootstrap is then employed on data unseen during model fitting to provide confidence intervals for the improvement for the average future patient (assuming future patients are similar to the patients in the trial). One may also test against a null scenario where the hypothesized personalization are not more useful than a standard of care. We demonstrate our method's promise on simulated data as well as on data from a randomized comparative trial investigating two treatments for depression.