On the challenges to learn from Natural Data Streams

TL;DR

This paper investigates the challenges of learning from Natural Data Streams, characterized by unbalanced, drifting, and correlated data, and evaluates various algorithms' classification performance in this complex setting.

Contribution

It provides a comprehensive analysis of multiple learning algorithms' performance on Natural Data Streams, highlighting the difficulties and potential solutions in this challenging scenario.

Findings

Algorithms face significant challenges with data drift and imbalance.

Incremental learning agents show varying adaptability to stream characteristics.

Experimental results identify promising approaches for real-world streaming data.

Abstract

In real-world contexts, sometimes data are available in form of Natural Data Streams, i.e. data characterized by a streaming nature, unbalanced distribution, data drift over a long time frame and strong correlation of samples in short time ranges. Moreover, a clear separation between the traditional training and deployment phases is usually lacking. This data organization and fruition represents an interesting and challenging scenario for both traditional Machine and Deep Learning algorithms and incremental learning agents, i.e. agents that have the ability to incrementally improve their knowledge through the past experience. In this paper, we investigate the classification performance of a variety of algorithms that belong to various research field, i.e. Continual, Streaming and Online Learning, that receives as training input Natural Data Streams. The experimental validation is carried out on three different datasets, expressly organized to replicate this challenging setting.