Is UWLS Really Better for Medical Research?

TL;DR

This study assesses whether UWLS is better than other methods for medical research meta-analyses, finding that RE provides more reliable results in small-sample settings.

Contribution

The study introduces a simulation-based evaluation of UWLS in medical meta-analyses, revealing limitations of prior model selection criteria in small-sample contexts.

Findings

RE estimator consistently produced more accurate standard errors than UWLS in small-sample settings.

Simulation results showed similar bias and efficiency across estimators, but RE outperformed UWLS in reliability.

FE estimator's performance was less clear, suggesting context-dependent choices for UWLS or FE.

Abstract

This study evaluates the performance of the Unrestricted Weighted Least Squares (UWLS) estimator in meta‐analyses of medical research. Using a large‐scale simulation approach, it addresses the limitations of model selection criteria in small‐sample contexts. Prior research using the Cochrane Database of Systematic Reviews (CDSR) reported that UWLS outperformed Random Effects (RE) and, in some cases, Fixed Effect (FE) estimators when assessed using AIC and BIC. However, we show that idiosyncratic characteristics of the CDSR datasets, notably their small sample sizes and weak‐signal settings (where key parameters are often small in magnitude), undermine the reliability of AIC and BIC for model selection. Accordingly, we simulate 108 000 datasets mirroring the original CDSR data. This allows us to know the true model parameters and evaluate the estimators more accurately. While all estimators performed similarly with respect to bias and efficiency, RE consistently produced more accurate standard errors than UWLS, making confidence intervals and hypothesis testing more reliable. The comparison with FE was less clear. We therefore recommend continued use of the RE estimator as a reliable general‐purpose approach for medical research, with the choice between UWLS and FE made in light of the likely extent of effect heterogeneity in the data.

Figures