MGSER-SAM: Memory-Guided Soft Experience Replay with Sharpness-Aware Optimization for Enhanced Continual Learning

TL;DR

MGSER-SAM introduces a memory replay-based continual learning algorithm that combines sharpness-aware optimization with strategies to mitigate task interference, significantly improving accuracy and reducing forgetting across benchmarks.

Contribution

It integrates the SAM optimizer into experience replay frameworks and addresses task conflict issues with soft logits and gradient alignment, advancing continual learning methods.

Findings

Outperforms baselines ER and DER++ in accuracy by 24.4% and 17.6%.

Achieves the lowest forgetting across multiple benchmarks.

Demonstrates consistent improvements in three continual learning scenarios.

Abstract

Deep neural networks suffer from the catastrophic forgetting problem in the field of continual learning (CL). To address this challenge, we propose MGSER-SAM, a novel memory replay-based algorithm specifically engineered to enhance the generalization capabilities of CL models. We first intergrate the SAM optimizer, a component designed for optimizing flatness, which seamlessly fits into well-known Experience Replay frameworks such as ER and DER++. Then, MGSER-SAM distinctively addresses the complex challenge of reconciling conflicts in weight perturbation directions between ongoing tasks and previously stored memories, which is underexplored in the SAM optimizer. This is effectively accomplished by the strategic integration of soft logits and the alignment of memory gradient directions, where the regularization terms facilitate the concurrent minimization of various training loss terms integral to the CL process. Through rigorous experimental analysis conducted across multiple benchmarks, MGSER-SAM has demonstrated a consistent ability to outperform existing baselines in all three CL scenarios. Comparing to the representative memory replay-based baselines ER and DER++, MGSER-SAM not only improves the testing accuracy by $24.4\%$ and $17.6\%$ respectively, but also achieves the lowest forgetting on each benchmark.