Sequential Drift Detection in Deep Learning Classifiers

TL;DR

This paper presents a method for detecting data drift in deep learning classifiers using neural network embeddings within a sequential decision framework, balancing false alarms and detection speed.

Contribution

It introduces a novel drift detection approach that controls false alarm rates and incorporates a loss function to optimize the tradeoff between false alarms and detection delay.

Findings

Effective control of false alarm rates in drift detection

Balanced detection speed and accuracy through a new loss function

Validated approach on multiple experimental datasets

Abstract

We utilize neural network embeddings to detect data drift by formulating the drift detection within an appropriate sequential decision framework. This enables control of the false alarm rate although the statistical tests are repeatedly applied. Since change detection algorithms naturally face a tradeoff between avoiding false alarms and quick correct detection, we introduce a loss function which evaluates an algorithm's ability to balance these two concerns, and we use it in a series of experiments.