Amicus Plato, sed magis amica veritas: There is a reproducibility crisis in COVID-19 Computational Fluid Dynamics studies

TL;DR

This paper reviews COVID-19 CFD studies, highlighting a significant reproducibility crisis with only 13% fully reproducible, and introduces a novel reproducibility index to improve scientific reliability.

Contribution

It systematically assesses the reproducibility of COVID-19 CFD studies and proposes a new index to quantify and enhance reproducibility standards.

Findings

Only 13% of studies achieved full reproducibility.

87% of studies were generally irreproducible.

A new reproducibility index was proposed to evaluate CFD studies.

Abstract

There is overwhelming evidence on SARS-CoV-2 Airborne Transmission (AT) in the ongoing COVID-19 outbreak. It is extraordinarily difficult, however, to deduce a generalized framework to assess the relative airborne transmission risk with respect to other modes. This is due to the complex biophysics entailed in such phenomena. Since the SARS outbreak in 2002, Computational Fluid Dynamics (CFD) has been one of the main tools scientists used to investigate AT of respiratory viruses. Now, CFD simulations produce intuitive and physically plausible colour-coded results that help scientists understand SARS-CoV-2 airborne transmission patterns. In addition to validation requirements, for any CFD model to be of epistemic value to the scientific community; it must be reproducible. In 2020, more than 45 published studies investigated SARS-CoV-2 airborne transmission in different scenarios using CFD. Here, I systematically review the published CFD studies of COVID-19 and discuss their reproducibility criteria with respect to the CFD modeling process. Using a Weighted Scoring Model (WSM), I propose a novel reproducibility index for CFD simulations of SARS-CoV-2 AT. The proposed index $(0 \leq R^{CFD}_j \leq 1)$ relies on three reproducibility criteria comprising 10 elements that represent the possibility of a CFD study (j) to be reproduced. Frustratingly, only 3 of 23 studies (13%) achieved full reproducibility index $(R^{CFD}_j\geq 0.9)$ while the remaining 87% were found generally irreproducible $(R^{CFD}_j<0.9)$. Without reproducible models, the scientific benefit of CFD simulations will remain hindered, fragmented and limited. In conclusion, I call the scientific community to apply more rigorous measures on reporting and publishing CFD simulations in COVID-19 research.