On the physical interpretation of a meta-analysis in the presence of heterogeneity and bias: from clinical trials to Mendelian randomization

TL;DR

This paper explores a physical analogy for understanding meta-analysis, especially under heterogeneity and bias, using the funnel plot and extending it to interpret statistical models and bias adjustments in evidence synthesis and causal inference.

Contribution

It formalizes the physical analogy of meta-analysis using estimating equations, linking bias correction methods to causal inference techniques.

Findings

Extended funnel plot aids intuitive understanding of meta-analysis

Formal analogy connects bias adjustment to causal inference methods

Provides a new perspective on interpreting heterogeneity and bias in meta-analysis

Abstract

The funnel plot is a graphical visualisation of summary data estimates from a meta-analysis, and is a useful tool for detecting departures from the standard modelling assumptions. Although perhaps not widely appreciated, a simple extension of the funnel plot can help to facilitate an intuitive interpretation of the mathematics underlying a meta-analysis at a more fundamental level, by equating it to determining the centre of mass of a physical system. We used this analogy, with some success, to explain the concepts of weighing evidence and of biased evidence to a young audience at the Cambridge Science Festival, without recourse to precise definitions or statistical formulae. In this paper we aim to formalise this analogy at a more technical level using the estimating equation framework: firstly, to help elucidate some of the basic statistical models employed in a meta-analysis and secondly, to forge new connections between bias adjustment in the evidence synthesis and causal inference literatures.