An AI-Based Public Health Data Monitoring System

TL;DR

This paper introduces a novel AI-driven ranking-based public health data monitoring system that significantly improves efficiency over traditional alert-based methods, enabling scalable, real-time analysis of large health datasets.

Contribution

It presents a new AI anomaly detection approach integrated into a ranking system for public health data monitoring, deployed at a national level for large-scale, real-time analysis.

Findings

54x increase in reviewer speed efficiency

Effective monitoring of up to 5 million data points daily

Significant improvement over traditional alert-based systems

Abstract

Public health experts need scalable approaches to monitor large volumes of health data (e.g., cases, hospitalizations, deaths) for outbreaks or data quality issues. Traditional alert-based monitoring systems struggle with modern public health data monitoring systems for several reasons, including that alerting thresholds need to be constantly reset and the data volumes may cause application lag. Instead, we propose a ranking-based monitoring paradigm that leverages new AI anomaly detection methods. Through a multi-year interdisciplinary collaboration, the resulting system has been deployed at a national organization to monitor up to 5,000,000 data points daily. A three-month longitudinal deployed evaluation revealed a significant improvement in monitoring objectives, with a 54x increase in reviewer speed efficiency compared to traditional alert-based methods. This work highlights the potential of human-centered AI to transform public health decision-making.