Ensuring Learning Guarantees on Concept Drift Detection with Statistical Learning Theory

TL;DR

This paper applies Statistical Learning Theory to formalize and ensure probabilistic learning guarantees in concept drift detection, addressing current gaps in theoretical assurances for data stream change detection.

Contribution

It introduces a methodology to meet SLT assumptions in concept drift detection, providing formal learning guarantees and evaluating existing algorithms under this framework.

Findings

Proposed a methodology to ensure SLT-based learning guarantees in CD

Assessed existing CD algorithms with the new methodology

Supports researchers in designing and evaluating CD algorithms

Abstract

Concept Drift (CD) detection intends to continuously identify changes in data stream behaviors, supporting researchers in the study and modeling of real-world phenomena. Motivated by the lack of learning guarantees in current CD algorithms, we decided to take advantage of the Statistical Learning Theory (SLT) to formalize the necessary requirements to ensure probabilistic learning bounds, so drifts would refer to actual changes in data rather than by chance. As discussed along this paper, a set of mathematical assumptions must be held in order to rely on SLT bounds, which are especially controversial in CD scenarios. Based on this issue, we propose a methodology to address those assumptions in CD scenarios and therefore ensure learning guarantees. Complementary, we assessed a set of relevant and known CD algorithms from the literature in light of our methodology. As main contribution, we expect this work to support researchers while designing and evaluating CD algorithms on different domains.