Success-Odds: An improved Win-Ratio

TL;DR

This paper introduces the success-odds, a modified win-ratio measure for comparing therapies in clinical trials with ordinal or dichotomous outcomes, addressing issues with ties and interpretability.

Contribution

It proposes the success-odds as a new, more intuitive measure for treatment effects that remains well-behaved in the presence of ties, along with hypothesis tests and confidence intervals.

Findings

Success-odds equals win-ratio when no ties are present.

Success-odds remains interpretable with ties, unlike win-ratio.

The paper discusses challenges in extending these measures to multiple treatments.

Abstract

Multiple and combined endpoints involving also non-normal outcomes appear in many clinical trials in various areas in medicine. In some cases, the outcome can be observed only on an ordinal or dichotomous scale. Then the success of two therapies is assessed by comparing the outcome of two randomly selected patients from the two therapy groups by 'better', 'equal' or 'worse'. These outcomes can be described by the probabilities $p^-=P(X<Y)$, $p_0=P(X=Y)$, and $p^+ =P(X~>~Y)$. For a clinician, however, these quantities are less intuitive. Therefore, Noether (1987) introduced the quantity $\lambda=p^+ / p^-$ assuming continuous distributions. The same quantity was used by Pocock et al. (2012) and by Wang and Pocock (2016) also for general non-normal outcomes and has been called 'win-ratio' $\lambda_{WR}$. Unlike Noether (1987), Wang and Pocock (2016) explicitly allowed for ties in the data. It is the aim of this manuscript to investigate the properties of $\lambda_{WR}$ in case of ties. It turns out that it has the strange property of becoming larger if the data are observed less accurately, i.e. include more ties. Thus, in case of ties, the win-ratio looses its appealing property to describe and quantify an intuitive and well interpretable treatment effect. Therefore, a slight modification of $\lambda_{WR} = \theta / (1-\theta)$ is suggested, namely the so-called 'success-odds' where $\theta=p^+ + \frac12 p_0$ is called a success of a therapy if $\theta>\frac12$. In the case of no ties, $\lambda_{SO}$ is identical to $\lambda_{WR}$. A test for the hypothesis $\lambda_{SO}=1$ and range preserving confidence intervals for $\lambda_{SO}$ are derived. By two counterexamples it is demonstrated that generalizations of both the win-ratio and the success-odds to more than two treatments or to stratified designs are not straightforward and need more detailed considerations.