Fine-grained Anomaly Detection in Sequential Data via Counterfactual Explanations

TL;DR

This paper introduces CFDet, a novel framework for fine-grained anomaly detection in sequential data that uses counterfactual explanations to identify specific anomalous entries, enhancing interpretability.

Contribution

The paper proposes a new interpretable framework, CFDet, combining Deep SVDD with counterfactual explanations for precise entry-level anomaly detection in sequences.

Findings

CFDet accurately detects anomalous entries in sequences.

CFDet outperforms existing methods in interpretability and precision.

Experimental results validate the effectiveness of CFDet on multiple datasets.

Abstract

Anomaly detection in sequential data has been studied for a long time because of its potential in various applications, such as detecting abnormal system behaviors from log data. Although many approaches can achieve good performance on anomalous sequence detection, how to identify the anomalous entries in sequences is still challenging due to a lack of information at the entry-level. In this work, we propose a novel framework called CFDet for fine-grained anomalous entry detection. CFDet leverages the idea of interpretable machine learning. Given a sequence that is detected as anomalous, we can consider anomalous entry detection as an interpretable machine learning task because identifying anomalous entries in the sequence is to provide an interpretation to the detection result. We make use of the deep support vector data description (Deep SVDD) approach to detect anomalous sequences and propose a novel counterfactual interpretation-based approach to identify anomalous entries in the sequences. Experimental results on three datasets show that CFDet can correctly detect anomalous entries.