Permutation-based multiple testing corrections for p-values and confidence intervals for cluster randomised trials

TL;DR

This paper develops and compares permutation-based methods for p-value correction and confidence interval derivation in cluster randomized trials with multiple outcomes, ensuring strong error control and improved efficiency.

Contribution

It introduces a novel permutation-based search procedure for confidence intervals and compares multiple correction methods, highlighting the Romano-Wolf procedure's advantages.

Findings

Romano-Wolf method maintains nominal error rates and coverage.

Permutation-based methods outperform no correction in efficiency.

The approach is effective under complex correlation structures.

Abstract

In this article, we derive and compare methods to derive \textit{p}-values and sets of confidence intervals with strong control of the family-wise error rates and coverage for estimates of treatment effects in cluster randomised trials with multiple outcomes. There are few methods for \textit{p}-value corrections and deriving confidence intervals, limiting their application in this setting. We discuss the methods of Bonferroni, Holm, and Romano \& Wolf (2005) and adapt them to cluster randomised trial inference using permutation-based methods with different test statistics. We develop a novel search procedure for confidence set limits using permutation tests to produce a set of confidence intervals under each method of correction. We conduct a simulation-based study to compare family-wise error rates, coverage of confidence sets, and the efficiency of each procedure in comparison to no correction using both model-based standard errors and permutation tests. We show that the Romano-Wolf type procedure has nominal error rates and coverage under non-independent correlation structures and is more efficient than the other methods in a simulation-based study. We also compare results from the analysis of a real-world trial.