Isconna: Streaming Anomaly Detection with Frequency and Patterns

TL;DR

Isconna is an online streaming anomaly detection method for dynamic networks that combines frequency and pattern analysis, using statistical sketches to efficiently identify anomalies in real-time data.

Contribution

It introduces a novel approach that integrates burst and pattern anomaly detection without maintaining detailed pattern snippets, enhancing efficiency and effectiveness.

Findings

Outperforms five state-of-the-art baselines on six real-world datasets.

Efficiently detects anomalies in streaming edge data with up to 20 million records.

Maintains low space complexity using count-min sketches.

Abstract

An edge stream is a common form of presentation of dynamic networks. It can evolve with time, with new types of nodes or edges being continuously added. Existing methods for anomaly detection rely on edge occurrence counts or compare pattern snippets found in historical records. In this work, we propose Isconna, which focuses on both the frequency and the pattern of edge records. The burst detection component targets anomalies between individual timestamps, while the pattern detection component highlights anomalies across segments of timestamps. These two components together produce three intermediate scores, which are aggregated into the final anomaly score. Isconna does not actively explore or maintain pattern snippets; it instead measures the consecutive presence and absence of edge records. Isconna is an online algorithm, it does not keep the original information of edge records; only statistical values are maintained in a few count-min sketches (CMS). Isconna's space complexity $O(rc)$ is determined by two user-specific parameters, the size of CMSs. In worst case, Isconna's time complexity can be up to $O(rc)$, but it can be amortized in practice. Experiments show that Isconna outperforms five state-of-the-art frequency- and/or pattern-based baselines on six real-world datasets with up to 20 million edge records.