Spectral Anomaly Detection in Very Large Graphs: Models, Noise, and Computational Complexity

TL;DR

This paper explores spectral methods for anomaly detection in large graphs, addressing modeling, noise robustness, and computational scalability to improve detection accuracy and efficiency.

Contribution

It introduces new models for network behavior, analyzes noise effects on spectral detection, and discusses scalable algorithms for large-scale graph analysis.

Findings

Spectral methods can effectively detect anomalies in large graphs.

Noise models impact detection performance, but fusion can mitigate effects.

Parallel eigensolvers are promising for scaling spectral algorithms.

Abstract

Anomaly detection in massive networks has numerous theoretical and computational challenges, especially as the behavior to be detected becomes small in comparison to the larger network. This presentation focuses on recent results in three key technical areas, specifically geared toward spectral methods for detection. We first discuss recent models for network behavior, and how their structure can be exploited for efficient computation of the principal eigenspace of the graph. In addition to the stochasticity of background activity, a graph of interest may be observed through a noisy or imperfect mechanism, which may hinder the detection process. A few simple noise models are discussed, and we demonstrate the ability to fuse multiple corrupted observations and recover detection performance. Finally, we discuss the challenges in scaling the spectral algorithms to large-scale high-performance computing systems, and present preliminary recommendations to achieve good performance with current parallel eigensolvers.