A Dynamic Retrieval-Augmented Generation System with Selective Memory and Remembrance

TL;DR

This paper presents Adaptive RAG Memory (ARM), a dynamic retrieval-augmented generation framework that uses selective remembrance and decay, improving efficiency and performance in knowledge retention and retrieval tasks.

Contribution

The paper introduces a novel dynamic memory system for RAG that enhances retrieval efficiency and interpretability without retraining, inspired by cognitive principles.

Findings

ARM achieves near state-of-the-art performance with fewer parameters.

Dynamic RAG with ARM improves retrieval speed and coverage.

The system offers a practical trade-off between quality, latency, and memory efficiency.

Abstract

We introduce \emph{Adaptive RAG Memory} (ARM), a retrieval-augmented generation (RAG) framework that replaces a static vector index with a \emph{dynamic} memory substrate governed by selective remembrance and decay. Frequently retrieved items are consolidated and protected from forgetting, while rarely used items gradually decay, inspired by cognitive consolidation and forgetting principles. On a lightweight retrieval benchmark, ARM reaches near state-of-the-art performance (e.g., NDCG@5 $\approx$ 0.940, Recall@5 $=1.000$) with only $\sim$22M parameters in the embedding layer, achieving the best efficiency among ultra-efficient models ($<$25M parameters). In addition, we compare static vs. dynamic RAG combinations across Llama 3.1 and GPT-4o. Llama 3.1 with static RAG achieves the highest key-term coverage (67.2\%) at moderate latency, while GPT-4o with a dynamic selective retrieval policy attains the fastest responses (8.2s on average) with competitive coverage (58.7\%). We further present an engineering optimization of the DynamicRAG implementation, making embedding weights configurable, adjustable at runtime, and robust to invalid settings. ARM yields competitive accuracy, self-regularizing memory growth, and interpretable retention dynamics without retraining the generator\color{black} and provides practical trade-off between quality, latency and memory efficiency for production and research RAG system.