Assessing model prediction performance for the expected cumulative number of recurrent events

TL;DR

This paper introduces a new scoring method to evaluate how well models predict the expected cumulative number of recurrent events, accounting for individual patient history and applicable with or without terminal events.

Contribution

The paper presents a novel score for recurrent event prediction that extends the Brier Score and provides theoretical decomposition and practical comparison tools.

Findings

The score can be asymptotically decomposed into mean squared error and an inseparability term.

Simulation studies demonstrate the score's effectiveness.

Application to hospitalisation data compares different predictive models.

Abstract

In a recurrent events setting, we introduce a new score designed to evaluate the prediction ability, for a given model, of the expected cumulative number of recurrent events. This score allows to take into account the individual history of a patient through its external covariates and can be seen as an extension of the Brier Score for single time to event data but works for recurrent events with or without a terminal event. Theoretical results are provided that show that under standard assumptions in a recurrent event context, our score can be asymptotically decomposed as the sum of the theoretical mean squared error between the model and the true expected cumulative number of recurrent events and an inseparability term that does not depend on the model. This decomposition is further illustrated on simulations studies. It is also shown that this score should be used in comparison with a null model, such as a nonparametric estimator that does not include the covariates. Finally, the score is applied for the prediction of hospitalisations on a dataset of patients suffering from atrial fibrillation and a comparison of the predictions performance of different models, such as the Cox model or the Aalen Model, is investigated.