Concept Drift Detection and Adaptation with Weak Supervision on Streaming Unlabeled Data

TL;DR

This paper introduces a novel two-fold method combining density-based clustering and weak supervision to detect and adapt to concept drift in high-dimensional, unlabeled streaming data such as text, video, and images, achieving over 90% precision.

Contribution

It presents a new approach that addresses virtual and real concept drift in complex streaming data domains using clustering and high-confidence labels for model adaptation.

Findings

Achieved over 90% precision in drift detection and adaptation.

Effective handling of high-dimensional, noisy streaming data.

Maintained performance over four years without human intervention.

Abstract

Concept drift in learning and classification occurs when the statistical properties of either the data features or target change over time; evidence of drift has appeared in search data, medical research, malware, web data, and video. Drift adaptation has not yet been addressed in high dimensional, noisy, low-context data such as streaming text, video, or images due to the unique challenges these domains present. We present a two-fold approach to deal with concept drift in these domains: a density-based clustering approach to deal with virtual concept drift (change in statistical properties of features) and a weak-supervision step to deal with real concept drift (change in statistical properties of target). Our density-based clustering avoids problems posed by the curse of dimensionality to create an evolving 'map' of the live data space, thereby addressing virtual drift in features. Our weak-supervision step leverages high-confidence labels (oracle or heuristic labels) to generate weighted training sets to generalize and update existing deep learners to adapt to changing decision boundaries (real drift) and create new deep learners for unseen regions of the data space. Our results show that our two-fold approach performs well with >90% precision in 2018, four years after initial deployment in 2014, without any human intervention.