Disagreement Concerning Effect-Measure Modification

TL;DR

This paper explores how the choice of effect measure and the consideration of opposite outcomes can influence the perceived modification of treatment effects, highlighting potential disagreements in effect interpretation.

Contribution

It demonstrates that effect-measure modification can differ depending on the outcome considered and provides guidance on reporting and interpreting effect measures.

Findings

Probability of agreement between effect measures is 5/6 under uniform risks.

Considering opposite outcomes can alter effect-measure modification conclusions.

Case studies illustrate the impact on real-world data analysis.

Abstract

Stratifying factors, like age and gender, can modify the effect of treatments and exposures on risk of a studied outcome. Several effect measures, including the relative risk, hazard ratio, odds ratio, and risk difference, can be used to measure this modification. It is known that choice of effect measure may determine the presence and direction of effect-measure modification. We show that considering the opposite outcome -- for example, recovery instead of death -- may similarly influence effect-measure modification. In fact, if the relative risk for the studied outcome and the relative risk for the opposite outcome agree about the direction of effect-measure modification, then so will the two cumulative hazard ratios, the risk difference, and the odds ratio. When risks are randomly sampled from the uniform (0,1) distribution, the probability of this happening is 5/6. Disagreement is probable enough that researchers considering one relative risk should also consider the other and further discussion if they disagree. (If possible, researchers should also report estimated risks.) We provide examples through case studies on HCV, COVID-19, and bankruptcy following melanoma treatment.