Sequential Outlier Detection based on Incremental Decision Trees

TL;DR

This paper presents an online outlier detection algorithm that uses incremental decision trees to model normal data and adaptively identify anomalies in a data stream, improving detection accuracy over existing methods.

Contribution

The paper introduces a novel incremental decision tree-based multi-modal density estimation approach for real-time outlier detection with adaptive thresholding.

Findings

Achieves performance comparable to the optimal fixed threshold in hindsight.

Demonstrates significant improvements over state-of-the-art methods.

Effectively models complex data distributions with growing decision trees.

Abstract

We introduce an online outlier detection algorithm to detect outliers in a sequentially observed data stream. For this purpose, we use a two-stage filtering and hedging approach. In the first stage, we construct a multi-modal probability density function to model the normal samples. In the second stage, given a new observation, we label it as an anomaly if the value of aforementioned density function is below a specified threshold at the newly observed point. In order to construct our multi-modal density function, we use an incremental decision tree to construct a set of subspaces of the observation space. We train a single component density function of the exponential family using the observations, which fall inside each subspace represented on the tree. These single component density functions are then adaptively combined to produce our multi-modal density function, which is shown to achieve the performance of the best convex combination of the density functions defined on the subspaces. As we observe more samples, our tree grows and produces more subspaces. As a result, our modeling power increases in time, while mitigating overfitting issues. In order to choose our threshold level to label the observations, we use an adaptive thresholding scheme. We show that our adaptive threshold level achieves the performance of the optimal pre-fixed threshold level, which knows the observation labels in hindsight. Our algorithm provides significant performance improvements over the state of the art in our wide set of experiments involving both synthetic as well as real data.