Health Sentinel: An AI Pipeline For Real-time Disease Outbreak Detection

TL;DR

Health Sentinel is an AI pipeline that automates real-time detection of disease outbreaks from online media, processing vast amounts of data to identify potential health events for timely public health response.

Contribution

The paper introduces a novel multi-stage AI pipeline combining ML and non-ML methods for extracting structured health event information from online articles.

Findings

Processed over 300 million news articles since April 2022.

Identified over 95,000 unique health events across India.

Shortlisted over 3,500 potential outbreaks for public health action.

Abstract

Early detection of disease outbreaks is crucial to ensure timely intervention by the health authorities. Due to the challenges associated with traditional indicator-based surveillance, monitoring informal sources such as online media has become increasingly popular. However, owing to the number of online articles getting published everyday, manual screening of the articles is impractical. To address this, we propose Health Sentinel. It is a multi-stage information extraction pipeline that uses a combination of ML and non-ML methods to extract events-structured information concerning disease outbreaks or other unusual health events-from online articles. The extracted events are made available to the Media Scanning and Verification Cell (MSVC) at the National Centre for Disease Control (NCDC), Delhi for analysis, interpretation and further dissemination to local agencies for timely intervention. From April 2022 till date, Health Sentinel has processed over 300 million news articles and identified over 95,000 unique health events across India of which over 3,500 events were shortlisted by the public health experts at NCDC as potential outbreaks.