Combined background information for meta-analysis evaluation

TL;DR

This paper proposes a method to evaluate the reliability of meta-analyses by examining the quality and heterogeneity of the observational studies they include, using background info and p-value plots.

Contribution

It introduces an approach to assess meta-analysis reliability through study quality and heterogeneity analysis, including p-value plot simulations and examples.

Findings

Identification of negative studies within meta-analyses

Simulation results of p-value plots

Multiple real-world examples of heterogeneity analysis

Abstract

Massive numbers of meta-analysis studies are being published. A Google Scholar search of "systematic review and meta-analysis" returns about 452k hits since 2014. The search was done on Jan 14, 2019. There is a need to have some way to judge the reliability of a positive claim made in a meta-analysis that uses observational studies. Our idea is to examine the quality of the observational studies used in the meta-analysis and to examine the heterogeneity of those studies. We provide background information and examples: a listing of negative studies, a simulation of p-value plots, and multiple examples of p-value plots.