A studentized permutation test for the treatment effect in individual participant data meta-analysis

TL;DR

This paper introduces a studentized permutation test for treatment effect analysis in individual participant data meta-analyses, offering improved control of type I error and shorter confidence intervals compared to existing methods.

Contribution

The paper proposes a novel studentized permutation test for IPD meta-analysis that enhances error control and interval efficiency over traditional approaches.

Findings

Permutation test controls type I error effectively.

Proposed intervals are often shorter than competitors.

Method performs well in simulation studies.

Abstract

Meta-analysis is a well-established tool used to combine data from several independent studies, each of which usually compares the effect of an experimental treatment with a control group. While meta-analyses are often performed using aggregated study summaries, they may also be conducted using individual participant data (IPD). Classical meta-analysis models may be generalized to handle continuous IPD by formulating them within a linear mixed model framework. IPD meta-analyses are commonly based on a small number of studies. Technically, inference for the overall treatment effect can be performed using Student-t approximation. However, as some approaches may not adequately control the type I error, Satterthwaite's or Kenward-Roger's method have been suggested to set the degrees-of-freedom parameter. The latter also adjusts the standard error of the treatment effect estimator. Nevertheless, these methods may be conservative. Since permutation tests are known to control the type I error and offer robustness to violations of distributional assumptions, we propose a studentized permutation test for the treatment effect based on permutations of standardized residuals across studies in IPD meta-analysis. Also, we construct confidence intervals for the treatment effect based on this test. The first interval is derived from the percentiles of the permutation distribution. The second interval is obtained by searching values closest to the effect estimate that are just significantly different from the true effect. In a simulation study, we demonstrate satisfactory performance of the proposed methods, often producing shorter confidence intervals compared with competitors.