Autonomous Concept Drift Threshold Determination

TL;DR

This paper introduces a dynamic threshold method for concept drift detection that adapts over time, proven to outperform fixed thresholds and validated through extensive experiments on diverse datasets.

Contribution

Proposes a theoretically grounded dynamic threshold algorithm that improves drift detection performance over traditional fixed thresholds.

Findings

Dynamic thresholds outperform fixed thresholds in drift detection.

The proposed method improves detection accuracy on synthetic and real datasets.

Extensive experiments validate the effectiveness of the adaptive approach.

Abstract

Existing drift detection methods focus on designing sensitive test statistics. They treat the detection threshold as a fixed hyperparameter, set once to balance false alarms and late detections, and applied uniformly across all datasets and over time. However, maintaining model performance is the key objective from the perspective of machine learning, and we observe that model performance is highly sensitive to this threshold. This observation inspires us to investigate whether a dynamic threshold could be provably better. In this paper, we prove that a threshold that adapts over time can outperform any single fixed threshold. The main idea of the proof is that a dynamic strategy, constructed by combining the best threshold from each individual data segment, is guaranteed to outperform any single threshold that apply to all segments. Based on the theorem, we propose a Dynamic Threshold Determination algorithm. It enhances existing drift detection frameworks with a novel comparison phase to inform how the threshold should be adjusted. Extensive experiments on a wide range of synthetic and real-world datasets, including both image and tabular data, validate that our approach substantially enhances the performance of state-of-the-art drift detectors.