Beyond Traditional Surveillance: Harnessing Expert Knowledge for Public Health Forecasting

TL;DR

This study highlights the importance of expert judgment in public health forecasting, especially when traditional surveillance is limited, by analyzing predictions and rationales from officials during a critical season.

Contribution

It demonstrates that expert knowledge can outperform models in certain predictions and proposes a systematic approach to incorporate human judgment into public health data systems.

Findings

Experts better predicted H3 dominance than models.

Experts assigned lower probabilities to implausible scenarios.

Rationales revealed reliance on historical patterns and experience.

Abstract

Downsizing the US public health workforce throughout 2025 amplifies potential risks during public health crises. Expert judgment from public health officials represents a vital information source, distinct from traditional surveillance infrastructure, that should be valued -- not discarded. Understanding how expert knowledge functions under constraints is essential for understanding the potential impact of reduced capacity. To explore expert forecasting capabilities, 114 public health officials at the 2024 CSTE workshop generated 103 predictions plus 102 rationales of peak hospitalizations and 114 predictions of influenza H3 versus H1 dominance in Pennsylvania for the 2024/25 season. We compared expert predictions to computational models and used rationales to analyze reasoning patterns using Latent Dirichlet Allocation. Experts better predicted H3 dominance and assigned lower probability to implausible scenarios than models. Expert rationales drew on historical patterns, pathogen interactions, vaccine data, and cumulative experience. Expert public health knowledge constitutes a critical data source that should be valued equally with traditional datasets. We recommend developing a national toolkit to systematically collect and analyze expert predictions and rationales, treating human judgment as quantifiable data alongside surveillance systems to enhance crisis response capabilities.