Continual Learning for non-stationary regression via Memory-Efficient Replay

TL;DR

This paper introduces a novel prototype-based generative replay framework for online task-free continual regression, effectively reducing forgetting and maintaining stable performance in dynamic data streams.

Contribution

It is the first to propose a prototype-based generative replay method specifically for continual regression in non-stationary environments.

Findings

Reduces forgetting in continual regression tasks

Provides more stable performance than existing methods

Effective on multiple benchmark datasets

Abstract

Data streams are rarely static in dynamic environments like Industry 4.0. Instead, they constantly change, making traditional offline models outdated unless they can quickly adjust to the new data. This need can be adequately addressed by continual learning (CL), which allows systems to gradually acquire knowledge without incurring the prohibitive costs of retraining them from scratch. Most research on continual learning focuses on classification problems, while very few studies address regression tasks. We propose the first prototype-based generative replay framework designed for online task-free continual regression. Our approach defines an adaptive output-space discretization model, enabling prototype-based generative replay for continual regression without storing raw data. Evidence obtained from several benchmark datasets shows that our framework reduces forgetting and provides more stable performance than other state-of-the-art solutions.