Improving the Presentation and Understanding of Risk Models

TL;DR

This paper emphasizes graphical methods like calibration plots and risk distribution curves to improve understanding of risk models, demonstrating how predictor number affects risk distribution and discrimination in clinical predictions.

Contribution

It introduces visual tools for better interpretation of risk models and shows how increasing predictors influences risk distribution and model discrimination.

Findings

Risk distribution curves become more dispersed with more predictors.

Distributions resemble lognormal when predictors interact multiplicatively.

Increased predictors improve discrimination measures.

Abstract

The key concepts (calibration, discrimination, and discordance) important in understanding and comparing risk models are best conveyed graphically. To illustrate this, models predicting death and acute kidney injury in a large cohort of PCI patients differing in the number of predictors included are presented. Calibration plots, often presented in the current literature, present the agreement between predicted and observed risk for deciles of risk. Risk distribution curves present the frequency of different levels of risk. Scatterplots of the risks assigned to individuals by different models show the discordance of the individual risk estimates. Increasing the number of predictors in these models produce increasingly disperse and progressively skewed risk distribution curves. These resemble the lognormal distributions expected when risk predictors interact multiplicatively. These changes in the risk distribution curves correlate with improved measures of discrimination.