TL;DR
This paper presents TFC-SR, a continual learning method inspired by human memory strategies, which improves neural network retention and performance on benchmarks by incorporating active recall probes into experience replay.
Contribution
Introduces TFC-SR, a novel continual learning approach that stabilizes past knowledge using active recall, inspired by human learning techniques, outperforming existing methods on standard benchmarks.
Findings
TFC-SR achieves 13.17% accuracy on Split CIFAR-100, surpassing standard experience replay.
Active recall probes significantly stabilize neural representations of past tasks.
TFC-SR performs well in memory-constrained environments, outperforming traditional replay methods.
Abstract
Deep neural networks often suffer from a critical limitation known as catastrophic forgetting, where performance on past tasks degrades after learning new ones. This paper introduces a novel continual learning approach inspired by human learning strategies like Active Recall, Deliberate Practice, and Spaced Repetition, named Task-Focused Consolidation with Spaced Recall (TFC-SR). TFC-SR enhances the standard experience replay framework with a mechanism we term the Active Recall Probe. It is a periodic, task-aware evaluation of the model's memory that stabilizes the representations of past knowledge. We test TFC-SR on the Split MNIST and the Split CIFAR-100 benchmarks against leading regularization-based and replay-based baselines. Our results show that TFC-SR performs significantly better than these methods. For instance, on the Split CIFAR-100, it achieves a final accuracy of 13.17%…
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
1. Intuitive and Simple Concept: The motivation drawn from human cognitive science—specifically Active Recall and Spaced Repetition—provides a compelling and intuitive narrative for the method. 2. Strong Performance in a Specific Regime: The paper demonstrates a significant performance advantage for TFC-SR in a memory-constrained setting on a challenging benchmark. 3. Insightful Ablation and Discussion: The ablation study on buffer capacity (Section 3.4) is a highlight of the paper.
1. Major Disconnect Between Proposed Mechanism and Likely Cause: The paper's central narrative is built around the cognitive science concepts of "Active Recall" and "Spaced Repetition." However, the authors' own analysis in Section 4 strongly suggests that the observed performance gain on Split CIFAR-100 is an architectural artifact related to Batch Normalization. 2. Inconsistent and Underwhelming Empirical Results: The experimental results are mixed and do not support a general claim of the met
Standard rehearsal approaches typically interleave batches of examples from the current task with batches retrieved from the memory buffer. In contrast, this work proposes replacing this regular and constant scheduling with an adaptive mechanism, where rehearsal is triggered only when accuracy on past tasks decreases. This is a sound and promising intuition: such an adaptive strategy could reduce unnecessary computation (e.g., memory retrieval and additional forward–backward passes) while mitiga
The paper has countless problems in its current form. The impression is that the paper is an exercise for a student to practice submitting to top conferences. In general, the writing is good, but it seems that an LLM did much of the work, as also corroborated by the statement placed by the authors on the last page. These are some of the major points I would like to point out: - The motivation is unclear. The authors make a proposal, which has some intuitive explanation inspired by how a student
- The authors motivate the idea of incorporating brain-inspired strategies into the training of Deep Learning methods. One of the challenges of current Deep Learning models is their limited ability to accumulate knowledge, something that happens naturally in humans. Human learning seems a natural source of inspiration for proposing new approaches to tackle this challenge. - Section 4 (Discussion) provides a clear explanation of the results and their comparison with other methods. Showing the con
- There is an evident lack of understanding of previous methods in the area of Continual Learning. For example, the paper's "Mixed-Batch" strategy, which it calls novel, is a common practice in the area. - Chrysakis, Aristotelis, and Marie-Francine Moens. "Online continual learning from imbalanced data." International Conference on Machine Learning. PMLR, 2020. - It is unclear how and when the Adaptive Active Recall is used during training. Algorithm 1 shows only how it is used during infer
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
