Non-Transitivity of the Win Ratio and the Area Under the Receiver Operating Characteristics Curve (AUC): a case for evaluating the strength of stochastic comparisons

TL;DR

This paper investigates the non-transitive behavior of the win ratio and AUC in randomized trials, emphasizing the importance of testing for non-transitivity to better understand stochastic relationships between treatments.

Contribution

It demonstrates the non-transitivity of WR and AUC, providing conditions for transitivity and non-transitivity, and highlights the significance of assessing stochastic relationship strength.

Findings

Non-transitivity observed in real trial data for WR of time-to-death

Conditions for transitivity and non-transitivity are specified

Studying non-transitivity enhances understanding of treatment comparisons

Abstract

The win ratio (WR) is a novel statistic used in randomized controlled trials that can account for hierarchies within event outcomes. In this paper we report and study the long-run non-transitive behavior of the win ratio and the closely related Area Under the Receiver Operating Characteristics Curve (AUC) and argue that their transitivity cannot be taken for granted. Crucially, traditional within-group statistics (i.e., comparison of means) are always transitive, while the WR can detect non-transitivity. Non-transitivity provides valuable information on the stochastic relationship between two treatment groups, which should be tested and reported. We specify the necessary conditions for transitivity, the sufficient conditions for non-transitivity, and demonstrate non-transitivity in a real-life large randomized controlled trial for the WR of time-to-death. Our results can be used to rule out or evaluate the possibility of non-transitivity and show the importance of studying the strength of stochastic relationships.