Meta-Analysis Without Normality: Estimating the True Effect Distribution with Penalized Gaussian Mixtures

TL;DR

This paper introduces a Penalized Gaussian Mixture framework for meta-analysis that accurately estimates the true effect distribution without assuming normality, especially useful for heterogeneous social science data.

Contribution

It develops a flexible, nonparametric method to recover the entire effect distribution, accommodating skewness and multimodality, improving over standard normal-based approaches.

Findings

PGM outperforms standard methods in large, heterogeneous meta-analyses.

It accurately estimates tail probabilities and density functions under non-normal effects.

The method is demonstrated to be practically useful with real environmental education data.

Abstract

Standard random-effects meta-analysis relies heavily on the assumption that the underlying true effects are normally distributed. In the social sciences, where evidence synthesis increasingly involves large, highly heterogeneous datasets, this assumption is often restrictive and unjustified. Misspecification of the random-effects distribution prevents the detection of asymmetry or multimodality, potentially leading to erroneous conclusions regarding the prevalence of adverse effects or the existence of specific subgroups. This paper introduces a Penalized Gaussian Mixture (PGM) framework designed to recover the entire probability density function of true effects without enforcing a rigid parametric shape. The method adapts to different non-normal scenarios, including skewed and multimodal distributions, while reducing to the normal case when supported by the data. A simulation study demonstrates that in large, highly heterogeneous meta-analyses, PGM yields substantially more accurate estimates of tail probabilities and the density function than standard methods when normality is violated, without substantially compromising efficiency under normality. An empirical application to environmental education data illustrates the practical utility of the method. The proposed framework provides researchers with a robust tool to move beyond simple summary statistics and characterize the complex nature of the true effect distribution in the real world.