Data augmentation on-the-fly and active learning in data stream classification

TL;DR

This paper introduces Augmented Queues, a novel online learning method that combines active learning, data augmentation, and multi-queue memory to improve data stream classification, especially under class imbalance and limited labels.

Contribution

The work presents a new online learning approach that efficiently balances classes and enhances data availability without extra memory, improving learning speed and quality.

Findings

Augmented Queues outperform existing methods in classification accuracy.

The method effectively handles class imbalance and limited labels.

It does not require additional memory for synthetic data generation.

Abstract

There is an emerging need for predictive models to be trained on-the-fly, since in numerous machine learning applications data are arriving in an online fashion. A critical challenge encountered is that of limited availability of ground truth information (e.g., labels in classification tasks) as new data are observed one-by-one online, while another significant challenge is that of class imbalance. This work introduces the novel Augmented Queues method, which addresses the dual-problem by combining in a synergistic manner online active learning, data augmentation, and a multi-queue memory to maintain separate and balanced queues for each class. We perform an extensive experimental study using image and time-series augmentations, in which we examine the roles of the active learning budget, memory size, imbalance level, and neural network type. We demonstrate two major advantages of Augmented Queues. First, it does not reserve additional memory space as the generation of synthetic data occurs only at training times. Second, learning models have access to more labelled data without the need to increase the active learning budget and / or the original memory size. Learning on-the-fly poses major challenges which, typically, hinder the deployment of learning models. Augmented Queues significantly improves the performance in terms of learning quality and speed. Our code is made publicly available.