Real Time Detection and Quantitative Analysis of Spurious Forgetting in Continual Learning

TL;DR

This paper introduces a quantitative framework for real-time detection and mitigation of spurious forgetting in continual learning, focusing on improving task alignment depth to enhance model robustness.

Contribution

It presents the first quantitative metrics and real-time detection methods for alignment depth, along with adaptive strategies to promote deep alignment and reduce forgetting.

Findings

Alignment is shallow, only covering 3-5 tokens, leading to vulnerability.

Promoting deep alignment improves robustness against forgetting.

Detection accuracy of 86.2-90.6% in experiments.

Abstract

Catastrophic forgetting remains a fundamental challenge in continual learning for large language models. Recent work revealed that performance degradation may stem from spurious forgetting caused by task alignment disruption rather than true knowledge loss. However, this work only qualitatively describes alignment, relies on post-hoc analysis, and lacks automatic distinction mechanisms. We introduce the shallow versus deep alignment framework, providing the first quantitative characterization of alignment depth. We identify that current task alignment approaches suffer from shallow alignment - maintained only over the first few output tokens (approximately 3-5) - making models vulnerable to forgetting. This explains why spurious forgetting occurs, why it is reversible, and why fine-tuning attacks are effective. We propose a comprehensive framework addressing all gaps: (1) quantitative metrics (0-1 scale) to measure alignment depth across token positions; (2) real-time detection methods for identifying shallow alignment during training; (3) specialized analysis tools for visualization and recovery prediction; and (4) adaptive mitigation strategies that automatically distinguish forgetting types and promote deep alignment. Extensive experiments on multiple datasets and model architectures (Qwen2.5-3B to Qwen2.5-32B) demonstrate 86.2-90.6% identification accuracy and show that promoting deep alignment improves robustness against forgetting by 3.3-7.1% over baselines.