Task-Sensitive Concept Drift Detector with Constraint Embedding

TL;DR

This paper introduces a task-sensitive semi-supervised drift detection method that effectively distinguishes between real and virtual drifts by leveraging label information during training and using constrained embedding representations.

Contribution

It presents a novel semi-supervised drift detection framework that utilizes constrained embeddings and can be adapted with various change detection methods, outperforming existing approaches.

Findings

Reliable detection of real drift demonstrated on nine benchmark datasets.

Outperforms state-of-the-art unsupervised drift detection methods.

Effectively ignores virtual drift where classification performance is unaffected.

Abstract

Detecting drifts in data is essential for machine learning applications, as changes in the statistics of processed data typically has a profound influence on the performance of trained models. Most of the available drift detection methods are either supervised and require access to the true labels during inference time, or they are completely unsupervised and aim for changes in distributions without taking label information into account. We propose a novel task-sensitive semi-supervised drift detection scheme, which utilizes label information while training the initial model, but takes into account that supervised label information is no longer available when using the model during inference. It utilizes a constrained low-dimensional embedding representation of the input data. This way, it is best suited for the classification task. It is able to detect real drift, where the drift affects the classification performance, while it properly ignores virtual drift, where the classification performance is not affected by the drift. In the proposed framework, the actual method to detect a change in the statistics of incoming data samples can be chosen freely. Experimental evaluation on nine benchmarks datasets, with different types of drift, demonstrates that the proposed framework can reliably detect drifts, and outperforms state-of-the-art unsupervised drift detection approaches.