Incorporating Expert Guidance in Epidemic Forecasting

TL;DR

This paper introduces a novel epidemic forecasting method that systematically incorporates expert guidance using Seldonian optimization, improving accuracy and safety in influenza predictions.

Contribution

It adapts the Seldonian framework to epidemic forecasting, enabling systematic inclusion of expert feedback to enhance model reliability and accuracy.

Findings

Bounded probability of undesirable outcomes

Reduced RMSE on test data by up to 17%

Effective incorporation of guidance improves forecast quality

Abstract

Forecasting influenza like illnesses (ILI) has rapidly progressed in recent years from an art to a science with a plethora of data-driven methods. While these methods have achieved qualified success, their applicability is limited due to their inability to incorporate expert feedback and guidance systematically into the forecasting framework. We propose a new approach leveraging the Seldonian optimization framework from AI safety and demonstrate how it can be adapted to epidemic forecasting. We study two types of guidance: smoothness and regional consistency of errors, where we show that by its successful incorporation, we are able to not only bound the probability of undesirable behavior to happen, but also to reduce RMSE on test data by up to 17%.