A Modified Net Reclassification Improvement Statistic

TL;DR

This paper proposes a modified version of the net reclassification improvement (NRI) statistic for binary risk models, addressing its issues of improper scoring and high false positive rates, demonstrated through prostate cancer data.

Contribution

It introduces a likelihood-based score residual guided modification of the continuous NRI to improve its statistical properties and interpretability.

Findings

The modified NRI behaves as a proper scoring function.

It reduces false positive rates in model comparison.

Application to prostate cancer data illustrates its practical utility.

Abstract

The continuous net reclassification improvement (NRI) statistic is a popular model change measure that was developed to assess the incremental value of new factors in a risk prediction model. Two prominent statistical issues identified in the literature call the utility of this measure into question: (1) it is not a proper scoring function and (2) it has a high false positive rate when testing whether new factors contribute to the risk model. For binary response regression models, these subjects are interrogated and a modification of the continuous NRI, guided by the likelihood-based score residual, is proposed to address these issues. Within a nested model framework, the modified NRI may be viewed as a distance measure between two risk models. An application of the modified NRI is illustrated using prostate cancer data.