Anomaly detection with superexperts under delayed feedback

TL;DR

This paper introduces a method for aggregating multiple unsupervised anomaly detection algorithms and incorporating delayed feedback, improving real-time cyber-attack detection with theoretical performance guarantees.

Contribution

The paper presents a novel aggregation framework for anomaly detection experts that handles delayed feedback and provides theoretical guarantees of near-optimal performance.

Findings

Aggregating models improves detection accuracy.

Incorporating feedback enhances performance.

The approach performs close to the best expert in theory.

Abstract

The increasing connectivity of data and cyber-physical systems has resulted in a growing number of cyber-attacks. Real-time detection of such attacks, through the identification of anomalous activity, is required so that mitigation and contingent actions can be effectively and rapidly deployed. We propose a new approach for aggregating unsupervised anomaly detection algorithms and incorporating feedback when it becomes available. We apply this approach to open-source real datasets and show that both aggregating models, which we call experts, and incorporating feedback significantly improve the performance. An important property of the proposed approaches is their theoretical guarantees that they perform close to the best superexpert, which can switch between the best performing experts, in terms of the cumulative average losses.