Statistical Evaluation of Spectral Methods for Anomaly Detection in Networks

TL;DR

This paper evaluates the statistical performance of spectral methods for anomaly detection in networks, focusing on Chi-square and L1 norm algorithms, through simulations and methodological improvements.

Contribution

It provides a comprehensive statistical analysis of spectral algorithms for network anomaly detection and extends their application to count networks.

Findings

Spectral methods can effectively detect small anomalous subgraphs.

Statistical properties of Chi-square and L1 algorithms are characterized.

Methodological improvements enhance detection performance.

Abstract

Monitoring of networks for anomaly detection has attracted a lot of attention in recent years especially with the rise of connected devices and social networks. This is of importance as anomaly detection could span a wide range of application, from detecting terrorist cells in counter-terrorism efforts to phishing attacks in social network circles. For this reason, numerous techniques for anomaly detection have been introduced. However, application of these techniques to more complex network models is hindered by various challenges such as the size of the network being investigated, how much apriori information is needed, the size of the anomalous graph, among others. A recent technique introduced by Miller et al, which relies on a spectral framework for anomaly detection, has the potential to address many of these challenges. In their discussion of the spectral framework, three algorithms were proposed that relied on the eigenvalues and eigenvectors of the residual matrix of a binary network. The authors demonstrated the ability to detect anomalous subgraphs that were less than 1% of the network size. However, to date, there is little work that has been done to evaluate the statistical performance of these algorithms. This study investigates the statistical properties of the spectral methods, specifically the Chi-square and L1 norm algorithm proposed by Miller. We will analyze the performance of the algorithm using simulated networks and also extend the method's application to count networks. Finally we will make some methodological improvements and recommendations to both algorithms.