Flexible and Efficient Drift Detection without Labels

TL;DR

This paper introduces a novel, label-free concept drift detection method using statistical process control, which outperforms existing techniques under computational constraints and can be integrated into semi-supervised frameworks.

Contribution

The paper presents a new flexible, efficient drift detection algorithm that operates without labels and demonstrates how it can be incorporated into a semi-supervised detection framework.

Findings

Outperforms previous methods in statistical power under computational constraints

Effective integration into semi-supervised drift detection frameworks

Promising results demonstrated through numerical simulations

Abstract

Machine learning models are being increasingly used to automate decisions in almost every domain, and ensuring the performance of these models is crucial for ensuring high quality machine learning enabled services. Ensuring concept drift is detected early is thus of the highest importance. A lot of research on concept drift has focused on the supervised case that assumes the true labels of supervised tasks are available immediately after making predictions. Controlling for false positives while monitoring the performance of predictive models used to make inference from extremely large datasets periodically, where the true labels are not instantly available, becomes extremely challenging. We propose a flexible and efficient concept drift detection algorithm that uses classical statistical process control in a label-less setting to accurately detect concept drifts. We show empirically that under computational constraints, our approach has better statistical power than previous known methods. Furthermore, we introduce a new semi-supervised drift detection framework to model the scenario of detecting drift (without labels) given prior detections, and show how our drift detection algorithm can be incorporated effectively into this framework. We demonstrate promising performance via numerical simulations.