Schedule-Robust Online Continual Learning
Ruohan Wang, Marco Ciccone, Giulia Luise, Andrew Yapp, Massimiliano, Pontil, Carlo Ciliberto

TL;DR
This paper introduces schedule-robustness in continual learning, proposing a new method that maintains performance across arbitrary data presentation schedules, and demonstrating significant improvements on image classification benchmarks.
Contribution
The paper defines schedule-robustness for continual learning and presents a novel approach that learns a schedule-robust predictor adaptable with replay data.
Findings
Outperforms existing methods on CL benchmarks
Demonstrates robustness against arbitrary data schedules
Achieves large margin improvements in image classification
Abstract
A continual learning (CL) algorithm learns from a non-stationary data stream. The non-stationarity is modeled by some schedule that determines how data is presented over time. Most current methods make strong assumptions on the schedule and have unpredictable performance when such requirements are not met. A key challenge in CL is thus to design methods robust against arbitrary schedules over the same underlying data, since in real-world scenarios schedules are often unknown and dynamic. In this work, we introduce the notion of schedule-robustness for CL and a novel approach satisfying this desirable property in the challenging online class-incremental setting. We also present a new perspective on CL, as the process of learning a schedule-robust predictor, followed by adapting the predictor using only replay data. Empirically, we demonstrate that our approach outperforms existing…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Sparse and Compressive Sensing Techniques
