An overview of methods for receiver operating characteristic analysis, with an application to SARS-CoV-2 vaccine-induced humoral responses in solid organ transplant recipients

TL;DR

This paper reviews various ROC analysis methods, discusses their trade-offs, and demonstrates their application in comparing SARS-CoV-2 vaccine responses in transplant recipients versus healthy controls.

Contribution

It provides an overview of ROC curve estimation techniques, offers practical guidance with R code, and applies these methods to real-world vaccine response data.

Findings

Different ROC estimation methods exhibit bias-variance trade-offs

Simulation studies illustrate methodological differences

Application to SARS-CoV-2 data compares transplant and control responses

Abstract

Receiver operating characteristic (ROC) analysis is a tool to evaluate the capacity of a numeric measure to distinguish between groups, often employed in the evaluation of diagnostic tests. Overall classification ability is sometimes crudely summarized by a single numeric measure such as the area under the empirical ROC curve. However, it may also be of interest to estimate the full ROC curve while leveraging assumptions regarding the nature of the data (parametric) or about the ROC curve directly (semiparametric). Although there has been recent interest in methods to conduct comparisons by way of stochastic ordering, nuances surrounding ROC geometry and estimation are not widely known in the broader scientific and statistical community. The overarching goals of this manuscript are to (1) provide an overview of existing frameworks for ROC curve estimation with examples, (2) offer intuition for and considerations regarding methodological trade-offs, and (3) supply sample R code to guide implementation. We utilize simulations to demonstrate the bias-variance trade-off across various methods. As an illustrative example, we analyze data from a recent cohort study in order to compare responses to SARS-CoV-2 vaccination between solid organ transplant recipients and healthy controls.