Equitable Discrimination in Survival Prediction: The Maximum Expected C-Index

TL;DR

This paper introduces the expected C-Index (ECI) as a tighter upperbound for survival prediction models, enabling more accurate discrimination measurement across subpopulations and revealing potential biases in existing models.

Contribution

It proposes the ECI and SUBECI metrics for better discrimination assessment in survival models, especially across diverse subpopulations, and applies them to kidney transplant survival prediction.

Findings

ECI for 10-year survival is 0.75, below the perfect score of 1.

Standard C-Index scores around 0.61, nearly halving the gap to the upperbound.

Subpopulation analysis reveals significant differences and potential biases.

Abstract

The C-Index measures the discrimination performance of survival prediction models. C-Index scores are often well below the upperbound of 1 that represents perfect prediction and closer to 0.5 as achieved by random prediction. Our first objective is to provide a tighter C-Index upperbound for proportional hazards models. Our second research objective is to measure discrimination performance for subpopulations, also relative to subpopulation specific upperbounds. We present the expected C-Index (ECI) as a tight upperbound for proportional hazards models. Moreover, we define the subpopulation C-Index (SUBCI) and a sub-population specific expected C-Index (SUBECI). The metrics are applied to predict death censored graft survival (DCGF) after deceased donor kidney transplant in the US with a Cox model using standard donor (KDPI), patient (EPTS), and (Class 1) mismatch predictors. With an ECI of 0.75 for 10-year survival, the new upperbound is well below 1. A C-Index performance around 0.61 or slightly above as commonly reported in literature and replicated in this study therefore closes almost half of the gap between the ECI and the 0.5 threshold. SUBECIs don't vary significantly from the overall ECI but there are substantial and significant differences among the SUBCIs. Extending this upperbound and C-Index to subpopulations enables to identify differences in discrimination upperbounds across subpopulations and in prediction model biases. A standard Cox model for DCGF in the US can be ethnically biased.