Estimating the Uncertainty of Neural Network Forecasts for Influenza Prevalence Using Web Search Activity

TL;DR

This paper demonstrates that Bayesian Neural Networks can effectively forecast influenza prevalence using web search data while providing reliable uncertainty estimates, improving decision-making during flu seasons.

Contribution

It introduces a method to incorporate both data and model uncertainty into neural network forecasts of influenza prevalence, maintaining accuracy.

Findings

BNNs provide reliable uncertainty estimates alongside forecasts.

Considering both data and model uncertainty improves forecast accuracy.

Recurrent BNNs outperform traditional models for horizons over 7 days.

Abstract

Influenza is an infectious disease with the potential to become a pandemic, and hence, forecasting its prevalence is an important undertaking for planning an effective response. Research has found that web search activity can be used to improve influenza models. Neural networks (NN) can provide state-of-the-art forecasting accuracy but do not commonly incorporate uncertainty in their estimates, something essential for using them effectively during decision making. In this paper, we demonstrate how Bayesian Neural Networks (BNNs) can be used to both provide a forecast and a corresponding uncertainty without significant loss in forecasting accuracy compared to traditional NNs. Our method accounts for two sources of uncertainty: data and model uncertainty, arising due to measurement noise and model specification, respectively. Experiments are conducted using 14 years of data for England, assessing the model's accuracy over the last 4 flu seasons in this dataset. We evaluate the performance of different models including competitive baselines with conventional metrics as well as error functions that incorporate uncertainty estimates. Our empirical analysis indicates that considering both sources of uncertainty simultaneously is superior to considering either one separately. We also show that a BNN with recurrent layers that models both sources of uncertainty yields superior accuracy for these metrics for forecasting horizons greater than 7 days.