# Meta-analysis models relaxing the random-effects normality assumption: methodological systematic review and simulation study

## TL;DR

This paper reviews and compares alternative statistical models for meta-analysis that relax the assumption of normally distributed study effects, finding that some non-normal models perform better in certain scenarios.

## Contribution

The study systematically reviews and simulates non-normal random-effects meta-analysis models, providing guidance on when to consider alternatives to the standard normal model.

## Key findings

- Mixture and semi-parametric models better handle latent clustering in study data.
- Normal models may give misleading results when heterogeneity or outliers are present.
- Alternative models show similar bias but differ in coverage probability, especially with high variance.

## Abstract

Random-effects meta-analysis is widely used for synthesizing the studies of a systematic review assuming a normal distribution for the study-specific effects. However, this assumption might not always be plausible. Alternative options have been suggested but not used in published meta-analyses.

We conducted a systematic review to identify articles that proposed alternative meta-analysis models assuming non-normal distributions for the random effects, such as skewed or semi-parametric distributions. Subsequently, we performed a simulation study to evaluate the performance of the identified models and to compare them with the normal model. We considered 22 scenarios varying the amount of random-effects variance, the number of included studies, and the shape of the true distribution: normal, skew-normal, and mixture of two normal distributions. For each scenario, we generated 1000 meta-analyses datasets. To investigate additional aspects of the alternative models, we also applied them at three extracted simulated datasets representing three scenarios with different true distributions.

We identified in total 27 articles suggesting 24 alternative models that can be classified into three broad categories: models based on long-tail and skewed distributions, on mixtures of distributions, and on Dirichlet process priors (DP). We compared 15 models in our simulation study implemented in the Frequentist or Bayesian framework. Results revealed small differences in bias between the different models but larger differences in the level of coverage probability. Scenarios with large random-effects variance, lead to more inaccurate estimates of the mean of the random-effects distribution. However, mixture and semi-parametric models revealed latent underlying clustering of studies and assisted to form subgroups of common characteristics. The three simulated datasets demonstrated similar patterns with the simulation study for the bias of the mean of the random-effects distribution.

Focusing only on the mean of the random-effects distribution in meta-analysis can be misleading when substantial heterogeneity is suspected or outliers are present. In such cases, identifying the factors that differentiate the studies and looking at the prediction intervals can be very informative. Based on our simulation, investigators could have the normal model as their starting point and consider alternative models as sensitivity analysis in view of seemingly non-normal data.

The online version contains supplementary material available at 10.1186/s12874-025-02658-3.