DEFENDER: Detecting and Forecasting Epidemics using Novel Data-analytics for Enhanced Response

TL;DR

This paper introduces DEFENDER, a system that uses social media and news data to detect, track, and forecast disease outbreaks, improving early warning and situational awareness.

Contribution

The paper presents a novel integrated system combining social media and news data with algorithms for outbreak detection, tracking, and forecasting, including a new location network creation technique.

Findings

Disease count tracking improved by 37% using multiple symptoms.

Forecasting symptom activity gained 5% accuracy over traditional models.

Location network created solely from Twitter data.

Abstract

In recent years social and news media have increasingly been used to explain patterns in disease activity and progression. Social media data, principally from the Twitter network, has been shown to correlate well with official disease case counts. This fact has been exploited to provide advance warning of outbreak detection, tracking of disease levels and the ability to predict the likelihood of individuals developing symptoms. In this paper we introduce DEFENDER, a software system that integrates data from social and news media and incorporates algorithms for outbreak detection, situational awareness, syndromic case tracking and forecasting. As part of this system we have developed a technique for creating a location network for any country or region based purely on Twitter data. We also present a disease count tracking approach which leverages counts from multiple symptoms, which was found to improve the tracking of diseases by 37 percent over a model that used only previous case data. Finally we attempt to forecast future levels of symptom activity based on observed user movement on Twitter, finding a moderate gain of 5 percent over a time series forecasting model.