Detecting Anomalies Using Rotated Isolation Forest

TL;DR

This paper introduces the Rotated Isolation Forest (RIF), a novel anomaly detection method that improves upon iForest and EIF by eliminating ghost clusters through dataset rotation, leading to more accurate anomaly detection.

Contribution

The paper proposes RIF, which uses random rotations to address ghost cluster issues in existing isolation forest methods, enhancing anomaly detection accuracy.

Findings

RIF outperforms iForest and EIF on synthetic datasets.

RIF achieves superior results on real-world datasets.

Rotating data effectively eliminates ghost clusters.

Abstract

The Isolation Forest (iForest), proposed by Liu, Ting, and Zhou at TKDE 2012, has become a prominent tool for unsupervised anomaly detection. However, recent research by Hariri, Kind, and Brunner, published in TKDE 2021, has revealed issues with iForest. They identified the presence of axis-aligned ghost clusters that can be misidentified as normal clusters, leading to biased anomaly scores and inaccurate predictions. In response, they developed the Extended Isolation Forest (EIF), which effectively solves these issues by eliminating the ghost clusters introduced by iForest. This enhancement results in improved consistency of anomaly scores and superior performance. We reveal a previously overlooked problem in the Extended Isolation Forest (EIF), showing that it is vulnerable to ghost inter-clusters between normal clusters of data points. In this paper, we introduce the Rotated Isolation Forest (RIF) algorithm which effectively addresses both the axis-aligned ghost clusters observed in iForest and the ghost inter-clusters seen in EIF. RIF accomplishes this by randomly rotating the dataset (using random rotation matrices and QR decomposition) before feeding it into the iForest construction, thereby increasing dataset variation and eliminating ghost clusters. Our experiments conclusively demonstrate that the RIF algorithm outperforms iForest and EIF, as evidenced by the results obtained from both synthetic datasets and real-world datasets.