Focusing Viral Risk Ranking Tool on Prediction

TL;DR

This study reanalyzed the Viral Risk Ranking tool by removing spillover-dependent factors, revealing that the original tool heavily relied on these factors for predicting zoonotic spillover risk, and suggesting improvements for future versions.

Contribution

The paper demonstrates that excluding spillover-dependent factors reduces the predictive accuracy of the Viral Risk Ranking tool, highlighting the need for non-spillover factors in risk assessment.

Findings

Original ranking scores correlated with high spillover risk.

Removing spillover-dependent factors decreased predictive accuracy.

Most spillover-dependent factors differed between human and non-human viruses.

Abstract

Preparing to rapidly respond to emerging infectious diseases is becoming ever more critical. "SpillOver: Viral Risk Ranking" is an open-source tool developed to evaluate novel wildlife-origin viruses for their risk of spillover from animals to humans and their risk of spreading in human populations. However, several of the factors used in the risk assessment are dependent on evidence of previous zoonotic spillover and/or sustained transmission in humans. Therefore, we performed a reanalysis of the "Ranking Comparison" after removing eight factors that require post-spillover knowledge and compared the adjusted risk rankings to the originals. The top 10 viruses as ranked by their adjusted scores also had very high original scores. However, the predictive power of the tool for whether a virus was a human virus or not as classified in the Spillover database deteriorated when these eight factors were removed. The area under the receiver operating characteristic curves (AUROC) for the original score, 0.94, decreased to 0.73 for the adjusted scores. Furthermore, we compared the mean and standard deviation of the human and non-human viruses at the factor level. Most of the excluded spillover-dependent factors had dissimilar means between the human and non-human virus groups compared to the non-spillover dependent factors, which frequently demonstrated similar means between the two groups with some exceptions. We concluded that the original formulation of the tool depended heavily on spillover-dependent factors to "predict" the risk of zoonotic spillover for a novel virus. Future iterations of the tool should take into consideration other non-spillover dependent factors and omit those that are spillover-dependent to ensure the tool is fit for purpose.