Label-invariant models for the analysis of meta-epidemiological data

TL;DR

This paper introduces label-invariant models for meta-epidemiological data analysis, allowing more flexible investigation of study heterogeneity related to methodological characteristics, demonstrated through analyses of Cochrane reviews.

Contribution

The paper proposes alternative label-invariant models that improve upon previous models by not forcing heterogeneity to be at least as large among certain study groups.

Findings

Heterogeneity variance is 88% greater in small trials.

Methodological flaws influence heterogeneity variances.

Models show wide confidence intervals for heterogeneity ratios.

Abstract

Rich meta-epidemiological data sets have been collected to explore associations between intervention effect estimates and study-level characteristics. Welton et al. proposed models for the analysis of meta-epidemiological data, but these models are restrictive because they force heterogeneity among studies with a particular characteristic to be at least as large as that among studies without the characteristic. In this paper we present alternative models that are invariant to the labels defining the two categories of studies. To exemplify the methods, we use a collection of meta-analyses in which the Cochrane Risk of Bias tool has been implemented. We first investigate the influence of small trial sample sizes (less than 100 participants), before investigating the influence of multiple methodological flaws (inadequate or unclear sequence generation, allocation concealment and blinding). We fit both the Welton et al. model and our proposed label-invariant model and compare the results. Estimates of mean bias associated with the trial characteristics and of between-trial variances are not very sensitive to the choice of model. Results from fitting a univariable model show that heterogeneity variance is, on average, 88% greater among trials with less than 100 participants. Based on a multivariable model, heterogeneity variance is, on average, 25% greater among trials with inadequate/unclear sequence generation, 51% greater among trials with inadequate/unclear blinding, and 23% lower among trials with inadequate/unclear allocation concealment, though the 95% intervals for these ratios are very wide. Our proposed label-invariant models for meta-epidemiological data analysis facilitate investigations of between-study heterogeneity attributable to certain study characteristics.