Odds Ratios are far from "portable": A call to use realistic models for effect variation in meta-analysis

TL;DR

This paper critiques the claim that odds ratios are portable across studies, demonstrating that their portability varies and emphasizing the importance of using realistic models to understand effect variation in meta-analyses.

Contribution

The authors empirically refute the claim that odds ratios are universally portable, highlighting errors in previous arguments and advocating for models that account for effect variation.

Findings

Odds ratios are not universally portable across different baseline risks.

Conversion methods from odds ratios to risk ratios can be biased.

Effect measure variation depends on study-specific factors like baseline risk.

Abstract

Objective: Recently Doi et al. argued that risk ratios should be replaced with odds ratios in clinical research. We disagreed, and empirically documented the lack of portability of odds ratios, while Doi et al. defended their position. In this response we highlight important errors in their position. Study Design and Setting: We counter Doi et al.'s arguments by further examining the correlations of odds ratios, and risk ratios, with baseline risks in 20,198 meta-analyses from the Cochrane Database of Systematic Reviews. Results: Doi et al.'s claim that odds ratios are portable is invalid because 1) their reasoning is circular: they assume a model under which the odds ratio is constant and show that under such a model the odds ratio is portable; 2) the method they advocate to convert odds ratios to risk ratios is biased; 3) their empirical example is readily-refuted by counter-examples of meta-analyses in which the risk ratio is portable but the odds ratio isn't; and 4) they fail to consider the causal determinants of meta-analytic inclusion criteria: Doi et al. mistakenly claim that variation in odds ratios with different baseline risks in meta-analyses is due to collider bias. Empirical comparison between the correlations of odds ratios, and risk ratios, with baseline risks show that the portability of odds ratios and risk ratios varies across settings. Conclusion: The suggestion to replace risk ratios with odds ratios is based on circular reasoning and a confusion of mathematical and empirical results. It is especially misleading for meta-analyses and clinical guidance. Neither the odds ratio nor the risk ratio is universally portable. To address this lack of portability, we reinforce our suggestion to report variation in effect measures conditioning on modifying factors such as baseline risk; understanding such variation is essential to patient-centered practice.