DenDrift: A Drift-Aware Algorithm for Host Profiling

TL;DR

DenDrift is a novel host profiling algorithm that enhances DenStream by incorporating drift detection techniques, improving robustness against behavioral changes in security monitoring scenarios.

Contribution

It introduces a drift-aware extension of DenStream using non-negative matrix factorization and Page-Hinckley test for improved host profiling accuracy.

Findings

Robust against abrupt, gradual, and incremental drifts

Effective on synthetic and industrial datasets

Improves reliability of host behavior profiles

Abstract

Detecting and reacting to unauthorized actions is an essential task in security monitoring. What make this task challenging are the large number and various categories of hosts and processes to monitor. To these we should add the lack of an exact definition of normal behavior for each category. Host profiling using stream clustering algorithms is an effective means of analyzing hosts' behaviors, categorizing them, and identifying atypical ones. However, unforeseen changes in behavioral data (i.e. concept drift) make the obtained profiles unreliable. DenStream is a well-known stream clustering algorithm, which can be effectively used for host profiling. This algorithm is an incremental extension of DBSCAN which is a non-parametric algorithm widely used in real-world clustering applications. Recent experimental studies indicate that DenStream is not robust against concept drift. In this paper, we present DenDrift as a drift-aware host profiling algorithm based on DenStream. DenDrift relies on non-negative matrix factorization for dimensionality reduction and Page-Hinckley test for drift detection. We have done experiments on both synthetic and industrial datasets and the results affirm the robustness of DenDrift against abrupt, gradual and incremental drifts.