Spatial Semantic Scan: Jointly Detecting Subtle Events and their Spatial Footprint

TL;DR

Spatial Semantic Scan (SCSS) is a novel method designed to rapidly detect locally emerging, spatially compact events in massive text streams, such as disease outbreaks, by combining semantic analysis with spatial neighborhood discovery.

Contribution

SCSS introduces an alternating optimization approach that jointly detects subtle events and their spatial footprints in real-time text streams, overcoming limitations of existing methods.

Findings

Successfully detected disease outbreaks in ED data

Outperformed existing event detection methods

Effective in identifying spatially localized events

Abstract

Many methods have been proposed for detecting emerging events in text streams using topic modeling. However, these methods have shortcomings that make them unsuitable for rapid detection of locally emerging events on massive text streams. We describe Spatially Compact Semantic Scan (SCSS) that has been developed specifically to overcome the shortcomings of current methods in detecting new spatially compact events in text streams. SCSS employs alternating optimization between using semantic scan to estimate contrastive foreground topics in documents, and discovering spatial neighborhoods with high occurrence of documents containing the foreground topics. We evaluate our method on Emergency Department chief complaints dataset (ED dataset) to verify the effectiveness of our method in detecting real-world disease outbreaks from free-text ED chief complaint data.