Risk-Adjusted learning curve assessment using comparative probability metrics

TL;DR

This paper introduces a risk-adjusted learning curve assessment method for surgical training that uses comparative probability metrics to evaluate performance improvements over time, especially for skewed outcomes like surgery durations.

Contribution

The paper presents a novel, interpretability-focused approach for surgical learning curve assessment that estimates clinical equivalence without fixed thresholds, accommodating skewed data distributions.

Findings

SLCA improves interpretability over CUSUM methods.

The method effectively assesses clinical noninferiority.

Case study demonstrates practical applicability.

Abstract

Surgical learning curves are graphical tools used to evaluate a trainee's progress in the early stages of their career and determine whether they have achieved proficiency after completing a specified number of surgeries. Cumulative sum (CUSUM) techniques are commonly used to assess learning curves due to their simplicity, but they face criticism for relying on fixed performance thresholds and lacking interpretability. This paper introduces a risk-adjusted surgical learning curve assessment (SLCA) method that focuses on estimation rather than hypothesis testing, as seen in CUSUM methods. The method is designed to accommodate right-skewed outcomes, such as surgery durations, characterized by the Weibull distribution. To evaluate the learning process, the SLCA approach estimates comparative probability metrics that assess the likelihood of a clinically important difference between the trainee's performance and a standard. Expecting improvement over time, we use weighted estimating equations to give greater weight to recent outcomes. Compared to CUSUM methods, SLCA offers enhanced interpretability, avoids reliance on externally defined performance levels, and emphasizes assessing clinical equivalence or noninferiority. We demonstrate the method's effectiveness through a colorectal surgery dataset case study and a numerical study.