Tracking changes using Kullback-Leibler divergence for the continual learning

TL;DR

This paper proposes a novel method using Kullback-Leibler divergence to monitor and predict concept drift in streaming data, enhancing continual learning by detecting changes in data distribution.

Contribution

It introduces a new approach for tracking probabilistic distribution changes in data streams using KL divergence, aiding in early concept drift detection.

Findings

KL divergence effectively predicts concept drift occurrence.

Method improves understanding of data distribution changes.

Results support application in real-time predictive maintenance.

Abstract

Recently, continual learning has received a lot of attention. One of the significant problems is the occurrence of \emph{concept drift}, which consists of changing probabilistic characteristics of the incoming data. In the case of the classification task, this phenomenon destabilizes the model's performance and negatively affects the achieved prediction quality. Most current methods apply statistical learning and similarity analysis over the raw data. However, similarity analysis in streaming data remains a complex problem due to time limitation, non-precise values, fast decision speed, scalability, etc. This article introduces a novel method for monitoring changes in the probabilistic distribution of multi-dimensional data streams. As a measure of the rapidity of changes, we analyze the popular Kullback-Leibler divergence. During the experimental study, we show how to use this metric to predict the concept drift occurrence and understand its nature. The obtained results encourage further work on the proposed methods and its application in the real tasks where the prediction of the future appearance of concept drift plays a crucial role, such as predictive maintenance.