Continual Learning with Diffusion-based Generative Replay for Industrial Streaming Data

TL;DR

This paper presents a diffusion-based generative replay method called DSG for continual learning in industrial streaming data, effectively mitigating catastrophic forgetting and adapting to data drift in IIoT environments.

Contribution

The paper introduces DSG, a novel knowledge distillation-based generative replay mechanism utilizing diffusion models for continual learning in industrial streaming data.

Findings

DSG outperforms state-of-the-art baselines in accuracy by 2.9% to 5.0%.

The method improves generator stability and data quality.

Experimental validation on multiple industrial datasets.

Abstract

The Industrial Internet of Things (IIoT) integrates interconnected sensors and devices to support industrial applications, but its dynamic environments pose challenges related to data drift. Considering the limited resources and the need to effectively adapt models to new data distributions, this paper introduces a Continual Learning (CL) approach, i.e., Distillation-based Self-Guidance (DSG), to address challenges presented by industrial streaming data via a novel generative replay mechanism. DSG utilizes knowledge distillation to transfer knowledge from the previous diffusion-based generator to the updated one, improving both the stability of the generator and the quality of reproduced data, thereby enhancing the mitigation of catastrophic forgetting. Experimental results on CWRU, DSA, and WISDM datasets demonstrate the effectiveness of DSG. DSG outperforms the state-of-the-art baseline in accuracy, demonstrating improvements ranging from 2.9% to 5.0% on key datasets, showcasing its potential for practical industrial applications.