Algorithmic Frameworks for the Detection of High Density Anomalies

TL;DR

This paper introduces non-parametric algorithmic frameworks for detecting high-density anomalies, which are deviations within normal data regions, addressing challenges posed by large or noisy datasets.

Contribution

It proposes novel unsupervised frameworks that leverage existing anomaly detection algorithms to effectively identify high-density anomalies.

Findings

IPP framework achieves the best detection performance

Frameworks outperform baseline algorithms on synthetic data

Effective in large and noisy datasets

Abstract

This study explores the concept of high-density anomalies. As opposed to the traditional concept of anomalies as isolated occurrences, high-density anomalies are deviant cases positioned in the most normal regions of the data space. Such anomalies are relevant for various practical use cases, such as misbehavior detection and data quality analysis. Effective methods for identifying them are particularly important when analyzing very large or noisy sets, for which traditional anomaly detection algorithms will return many false positives. In order to be able to identify high-density anomalies, this study introduces several non-parametric algorithmic frameworks for unsupervised detection. These frameworks are able to leverage existing underlying anomaly detection algorithms and offer different solutions for the balancing problem inherent in this detection task. The frameworks are evaluated with both synthetic and real-world datasets, and are compared with existing baseline algorithms for detecting traditional anomalies. The Iterative Partial Push (IPP) framework proves to yield the best detection results.