Unsupervised Anomalous Data Space Specification

TL;DR

This paper introduces a method to precompute a complete specification of an algorithm's behavior, transforming anomaly detection algorithms into tools that can generate comprehensive anomaly specifications from minimal training data.

Contribution

It presents a novel approach to convert anomaly detection algorithms into specification-generating tools, enabling quick, comprehensive anomaly descriptions from small datasets.

Findings

Precomputed specifications can be generated in constant time.

Specifications allow assessing how close an anomaly is to normal.

Method has wide applicability to algorithms with recoverable runtime behavior.

Abstract

Computer algorithms are written with the intent that when run they perform a useful function. Typically any information obtained is unknown until the algorithm is run. However, if the behavior of an algorithm can be fully described by precomputing just once how this algorithm will respond when executed on any input, this precomputed result provides a complete specification for all solutions in the problem domain. We apply this idea to a previous anomaly detection algorithm, and in doing so transform it from one that merely detects individual anomalies when asked to discover potentially anomalous values, into an algorithm also capable of generating a complete specification for those values it would deem to be anomalous. This specification is derived by examining no more than a small training data, can be obtained in very small constant time, and is inherently far more useful than results obtained by repeated execution of this tool. For example, armed with such a specification one can ask how close an anomaly is to being deemed normal, and can validate this answer not by exhaustively testing the algorithm but by examining if the specification so generated is indeed correct. This powerful idea can be applied to any algorithm whose runtime behavior can be recovered from its construction and so has wide applicability.