Permutation inference methods for multivariate meta-analysis

TL;DR

This paper introduces permutation-based inference methods for multivariate meta-analysis that provide exact joint and marginal inferences, overcoming limitations of traditional methods especially with small sample sizes.

Contribution

The paper develops permutation inference techniques for multivariate meta-analysis that do not rely on large sample approximations, offering more accurate confidence regions and intervals.

Findings

Permutation methods achieve accurate coverage in simulations.

Standard methods show undercoverage in small samples.

Applications demonstrate practical effectiveness of the proposed methods.

Abstract

Multivariate meta-analysis is gaining prominence in evidence synthesis research because it enables simultaneous synthesis of multiple correlated outcome data, and random-effects models have generally been used for addressing between-studies heterogeneities. However, coverage probabilities of confidence regions or intervals for standard inference methods for random-effects models (e.g., restricted maximum likelihood estimation) cannot retain their nominal confidence levels in general, especially when the number of synthesized studies is small because their validities depend on large sample approximations. In this article, we provide permutation-based inference methods that enable exact joint inferences for average outcome measures without large sample approximations. We also provide accurate marginal inference methods under general settings of multivariate meta-analyses. We propose effective approaches for permutation inferences using optimal weighting based on the efficient score statistic. The effectiveness of the proposed methods is illustrated via applications to bivariate meta-analyses of diagnostic accuracy studies for airway eosinophilia in asthma and a network meta-analysis for antihypertensive drugs on incident diabetes, as well as through simulation experiments. In numerical evaluations performed via simulations, our methods generally provided accurate confidence regions or intervals under a broad range of settings, whereas the current standard inference methods exhibited serious undercoverage properties.