PVBF: A Framework for Mitigating Parameter Variation Imbalance in Online Continual Learning

TL;DR

This paper introduces PVBF, a framework that mitigates parameter variation imbalance in online continual learning, significantly reducing prediction bias and enhancing accuracy through novel correlation-based and layer-wise balancing techniques.

Contribution

The paper proposes PVBF, a novel framework that addresses parameter variation imbalance in ER-based online continual learning, improving performance by balancing parameter updates across layers and tasks.

Findings

PVBF reduces prediction bias in online continual learning.

PVBF achieves up to 47% higher accuracy than existing methods.

The framework effectively balances parameter variations across layers and tasks.

Abstract

Online continual learning (OCL), which enables AI systems to adaptively learn from non-stationary data streams, is commonly achieved using experience replay (ER)-based methods that retain knowledge by replaying stored past during training. However, these methods face challenges of prediction bias, stemming from deviations in parameter update directions during task transitions. This paper identifies parameter variation imbalance as a critical factor contributing to prediction bias in ER-based OCL. Specifically, using the proposed parameter variation evaluation method, we highlight two types of imbalance: correlation-induced imbalance, where certain parameters are disproportionately updated across tasks, and layer-wise imbalance, where output layer parameters update faster than those in preceding layers. To mitigate the above imbalances, we propose the Parameter Variation Balancing Framework (PVBF), which incorporates: 1) a novel method to compute parameter correlations with previous tasks based on parameter variations, 2) an encourage-and-consolidate (E&C) method utilizing parameter correlations to perform gradient adjustments across all parameters during training, 3) a dual-layer copy weights with reinit (D-CWR) strategy to slowly update output layer parameters for frequently occuring sample categories. Experiments on short and long task sequences demonstrate that PVBF significantly reduces prediction bias and improves OCL performance, achieving up to 47\% higher accuracy compared to existing ER-based methods.