A Typology of Data Anomalies

TL;DR

This paper introduces a comprehensive and practical typology of data anomalies that clarifies their types, aiding in the evaluation of anomaly detection algorithms and enhancing understanding of anomaly concepts.

Contribution

It presents a general, applicable typology of data anomalies that improves upon previous models by providing clear definitions and evaluation frameworks.

Findings

Provides a tangible definition of anomaly types

Facilitates evaluation of anomaly detection algorithms

Serves as an analytical tool for anomaly analysis

Abstract

Anomalies are cases that are in some way unusual and do not appear to fit the general patterns present in the dataset. Several conceptualizations exist to distinguish between different types of anomalies. However, these are either too specific to be generally applicable or so abstract that they neither provide concrete insight into the nature of anomaly types nor facilitate the functional evaluation of anomaly detection algorithms. With the recent criticism on 'black box' algorithms and analytics it has become clear that this is an undesirable situation. This paper therefore introduces a general typology of anomalies that offers a clear and tangible definition of the different types of anomalies in datasets. The typology also facilitates the evaluation of the functional capabilities of anomaly detection algorithms and as a framework assists in analyzing the conceptual levels of data, patterns and anomalies. Finally, it serves as an analytical tool for studying anomaly types from other typologies.