'Beating the news' with EMBERS: Forecasting Civil Unrest using Open Source Indicators

TL;DR

EMBERS is an automated system that forecasts civil unrest in Latin America using open source data, demonstrating successful predictions during major protests and outperforming baseline methods.

Contribution

This paper introduces EMBERS, a continuous forecasting system leveraging diverse open source indicators, with real-time evaluation and interpretability features.

Findings

Successfully forecasted the June 2013 protests in Brazil

Outperformed baseline prediction methods

Provides an interpretable prediction audit trail

Abstract

We describe the design, implementation, and evaluation of EMBERS, an automated, 24x7 continuous system for forecasting civil unrest across 10 countries of Latin America using open source indicators such as tweets, news sources, blogs, economic indicators, and other data sources. Unlike retrospective studies, EMBERS has been making forecasts into the future since Nov 2012 which have been (and continue to be) evaluated by an independent T&E team (MITRE). Of note, EMBERS has successfully forecast the uptick and downtick of incidents during the June 2013 protests in Brazil. We outline the system architecture of EMBERS, individual models that leverage specific data sources, and a fusion and suppression engine that supports trading off specific evaluation criteria. EMBERS also provides an audit trail interface that enables the investigation of why specific predictions were made along with the data utilized for forecasting. Through numerous evaluations, we demonstrate the superiority of EMBERS over baserate methods and its capability to forecast significant societal happenings.