SR-CIS: Self-Reflective Incremental System with Decoupled Memory and Reasoning

TL;DR

SR-CIS is a novel incremental learning system inspired by human memory, combining fast inference and slow deliberation with decoupled memory modules, achieving stable learning with limited storage.

Contribution

It introduces a self-reflective system with decoupled memory and reasoning modules, utilizing task-specific memory and anomaly detection for efficient incremental learning.

Findings

Outperforms existing baselines on standard benchmarks.

Achieves stable incremental memory with limited storage.

Balances plasticity and stability effectively.

Abstract

The ability of humans to rapidly learn new knowledge while retaining old memories poses a significant challenge for current deep learning models. To handle this challenge, we draw inspiration from human memory and learning mechanisms and propose the Self-Reflective Complementary Incremental System (SR-CIS). Comprising the deconstructed Complementary Inference Module (CIM) and Complementary Memory Module (CMM), SR-CIS features a small model for fast inference and a large model for slow deliberation in CIM, enabled by the Confidence-Aware Online Anomaly Detection (CA-OAD) mechanism for efficient collaboration. CMM consists of task-specific Short-Term Memory (STM) region and a universal Long-Term Memory (LTM) region. By setting task-specific Low-Rank Adaptive (LoRA) and corresponding prototype weights and biases, it instantiates external storage for parameter and representation memory, thus deconstructing the memory module from the inference module. By storing textual descriptions of images during training and combining them with the Scenario Replay Module (SRM) post-training for memory combination, along with periodic short-to-long-term memory restructuring, SR-CIS achieves stable incremental memory with limited storage requirements. Balancing model plasticity and memory stability under constraints of limited storage and low data resources, SR-CIS surpasses existing competitive baselines on multiple standard and few-shot incremental learning benchmarks.