Statistical reproducibility of meta-analysis research claims for medical mask use in community settings to prevent COVID infection

TL;DR

This study critically evaluates the statistical reproducibility of meta-analyses claiming that medical masks prevent COVID infection in community settings, finding little evidence to support such benefits.

Contribution

It applies p-value plotting to assess the reproducibility of existing meta-analyses on mask efficacy, highlighting the lack of consistent evidence.

Findings

No evidence of mask benefit in six studies

Claims of no benefit were reproducible

Claims of benefit were not reproducible

Abstract

The coronavirus pandemic (COVID) has been an exceptional test of current scientific evidence that inform and shape policy. Many US states, cities, and counties implemented public orders for mask use on the notion that this intervention would delay and flatten the epidemic peak and largely benefit public health outcomes. P-value plotting was used to evaluate statistical reproducibility of meta-analysis research claims of a benefit for medical (surgical) mask use in community settings to prevent COVID infection. Eight studies (seven meta-analyses, one systematic review) published between 1 January 2020 and 7 December 2022 were evaluated. Base studies were randomized control trials with outcomes of medical diagnosis or laboratory-confirmed diagnosis of viral (Influenza or COVID) illness. Self-reported viral illness outcomes were excluded because of awareness bias. No evidence was observed for a medical mask use benefit to prevent viral infections in six p-value plots (five meta-analyses and one systematic review). Research claims of no benefit in three meta-analyses and the systematic review were reproduced in p-value plots. Research claims of a benefit in two meta-analyses were not reproduced in p-value plots. Insufficient data were available to construct p-value plots for two meta-analyses because of overreliance on self-reported outcomes. These findings suggest a benefit for medical mask use in community settings to prevent viral, including COVID infection, is unproven.