# What can we learn from 1,000 meta-analyses across 10 different disciplines?

## TL;DR

This study compares meta-analysis practices across 10 disciplines, revealing significant differences in methodology and highlighting common shortcomings.

## Contribution

The novelty lies in systematically analyzing 1,000 meta-analyses across disciplines to identify patterns and gaps in methodological practices.

## Key findings

- Disciplines differ in meta-analysis size, effect sizes per study, and types of effect sizes used.
- Many meta-analyses fail to account for statistical dependence when using multiple effect sizes from the same study.
- Advanced methods like multilevel models are rarely used, and meta-regression is underutilized despite sufficient sample sizes.

## Abstract

This study analyzes 1,000 meta-analyses drawn from 10 disciplines—including medicine, psychology, education, biology, and economics—to document and compare methodological practices across fields. We find large differences in the size of meta-analyses, the number of effect sizes per study, and the types of effect sizes used. Disciplines also vary in their use of unpublished studies, the frequency and type of tests for publication bias, and whether they attempt to correct for it. Notably, many meta-analyses include multiple effect sizes from the same study, yet fail to account for statistical dependence in their analyses. We document the limited use of advanced methods—such as multilevel models and cluster-adjusted standard errors—that can accommodate dependent data structures. Correlations are frequently used as effect sizes in some disciplines, yet researchers often fail to address the methodological issues this introduces, including biased weighting and misleading tests for publication bias. We also find that meta-regression is underutilized, even when sample sizes are large enough to support it. This work serves as a resource for researchers conducting their first meta-analyses, as a benchmark for researchers designing simulation experiments, and as a reference for applied meta-analysts aiming to improve their methodological practices.