A note on sampling biases in the Bangladesh mask trial

TL;DR

This paper re-analyzes a Bangladesh mask trial, highlighting how sampling biases and unblinded staff behavior can distort the interpretation of intervention effects, especially when using rate-based outcomes.

Contribution

It demonstrates the importance of non-parametric paired tests and reveals biases introduced by unblinded staff in cluster trials.

Findings

Behavioral outcomes like physical distancing are highly significant.

The primary COVID-19 outcome is not significant when re-analyzed.

Staff behavior significantly biases the treatment-control comparison.

Abstract

A recent cluster trial in Bangladesh randomized 600 villages into 300 treatment/control pairs, to evaluate the impact of an intervention to increase mask-wearing. Data was analyzed in a generalized linear model and significance asserted with parametric tests for the rate of the primary outcome (symptomatic and seropositive for COVID-19) between treatment and control villages. In this short note we re-analyze the data from this trial using standard non-parametric paired statistics tests on treatment/village pairs. With this approach, we find that behavioral outcomes like physical distancing are highly significant, while the primary outcome of the study is not. Importantly, we find that the behavior of unblinded staff when enrolling study participants is one of the most highly significant differences between treatment and control groups, contributing to a significant imbalance in denominators between treatment and control groups. The potential bias leading to this imbalance suggests caution is warranted when evaluating rates rather than counts. More broadly, the significant impacts on staff and participant behavior urge caution in interpreting small differences in the study outcomes that depended on survey response.