Getting it right: Methods for risk ratios and risk differences cluster randomized trials with a small number of clusters

TL;DR

This paper evaluates bias-corrected methods for inference on risk ratios and differences in small cluster randomized trials, emphasizing the importance of appropriate variance estimation and t-statistics for accurate results.

Contribution

It extends the evaluation of bias-correction methods to risk ratios and differences in small CRTs across various models and scenarios, filling a gap in existing research.

Findings

Bias-corrected standard errors improve inference accuracy.

Using t-statistics reduces Type I error inflation.

Performance varies across models and scenarios.

Abstract

Most cluster randomized trials (CRTs) randomize fewer than 30-40 clusters in total. When performing inference for such ``small'' CRTs, it is important to use methods that appropriately account for the small sample size. When the generalized estimating equations (GEE) approach is used for analysis of ``small'' CRTs, the robust variance estimator from GEE is biased downward and therefore bias-corrected standard errors should be used. Moreover, in order to avoid inflated Type I error, an appropriate bias-corrected standard error should be paired with the t- rather than Z-statistic when making inference about a single-parameter intervention effect. Although several bias-correction methods (including Kauermann and Carroll (KC), Mancl and DeRouen (MD), Morel, Bokossa, and Neerchal (MBN), and the average of KC and MD (AVG)) have been evaluated for inference for odds ratios, their finite-sample behavior in ``small'' CRTs with few clusters has not been thoroughly investigated for risk ratios and risk differences. The current article aims to fill the gap by including analysis via binomial, Poisson and Gaussian models and for a broad spectrum of scenarios. Analysis is via binomial and Poisson models (using log and identity link for risk and differences measures, respectively). We additionally explore the use of Gaussian models with identity link for risk differences and adopt the "modified" approach for analysis with misspecified Poisson and Gaussian models. We consider a broad spectrum of scenarios including for rare outcomes, small cluster sizes, high intracluster correlations (ICCs), and high coefficients of variation (CVs) of cluster size.