Improved Anomaly Detection by Using the Attention-Based Isolation Forest

TL;DR

The paper introduces Attention-Based Isolation Forest (ABIForest), which enhances anomaly detection by integrating an attention mechanism into the traditional Isolation Forest, leading to improved performance demonstrated through experiments.

Contribution

It presents the first modification of Isolation Forest that incorporates attention weights via Nadaraya-Watson regression without using gradient-based training.

Findings

ABIForest outperforms standard Isolation Forest in experiments.

The method effectively assigns attention weights based on data instances.

Numerical results show improved anomaly detection accuracy.

Abstract

A new modification of Isolation Forest called Attention-Based Isolation Forest (ABIForest) for solving the anomaly detection problem is proposed. It incorporates the attention mechanism in the form of the Nadaraya-Watson regression into the Isolation Forest for improving solution of the anomaly detection problem. The main idea underlying the modification is to assign attention weights to each path of trees with learnable parameters depending on instances and trees themselves. The Huber's contamination model is proposed to be used for defining the attention weights and their parameters. As a result, the attention weights are linearly depend on the learnable attention parameters which are trained by solving the standard linear or quadratic optimization problem. ABIForest can be viewed as the first modification of Isolation Forest, which incorporates the attention mechanism in a simple way without applying gradient-based algorithms. Numerical experiments with synthetic and real datasets illustrate outperforming results of ABIForest. The code of proposed algorithms is available.