Don't Look Back in Anger: MAGIC Net for Streaming Continual Learning with Temporal Dependence

TL;DR

MAGIC Net is a streaming continual learning model that combines recurrent neural networks with architectural strategies to handle temporal dependence, concept drift, and catastrophic forgetting in data streams, enabling continuous online learning.

Contribution

It introduces MAGIC Net, a novel architecture that integrates CL strategies with RNNs, allowing online learning with memory expansion and knowledge retention in streaming data.

Findings

Improves adaptation to new concepts in data streams

Limits memory usage effectively

Mitigates catastrophic forgetting during learning

Abstract

Concept drift, temporal dependence, and catastrophic forgetting represent major challenges when learning from data streams. While Streaming Machine Learning and Continual Learning (CL) address these issues separately, recent efforts in Streaming Continual Learning (SCL) aim to unify them. In this work, we introduce MAGIC Net, a novel SCL approach that integrates CL-inspired architectural strategies with recurrent neural networks to tame temporal dependence. MAGIC Net continuously learns, looks back at past knowledge by applying learnable masks over frozen weights, and expands its architecture when necessary. It performs all operations online, ensuring inference availability at all times. Experiments on synthetic and real-world streams show that it improves adaptation to new concepts, limits memory usage, and mitigates forgetting.