CLAG: Adaptive Memory Organization via Agent-Driven Clustering for Small Language Model Agents

TL;DR

CLAG is a memory organization framework for small language model agents that uses clustering to improve knowledge retention, reduce interference, and enhance answer quality.

Contribution

It introduces an agent-driven clustering memory system that autonomously organizes and retrieves relevant information, improving performance over traditional memory methods.

Findings

CLAG improves answer quality across multiple QA datasets.

It reduces cross-topic interference in memory retrieval.

CLAG remains lightweight and efficient for small language models.

Abstract

Large language model agents heavily rely on external memory to support knowledge reuse and complex reasoning tasks. Yet most memory systems store experiences in a single global retrieval pool which can gradually dilute or corrupt stored knowledge. This problem is especially pronounced for small language models (SLMs), which are highly vulnerable to irrelevant context. We introduce CLAG, a CLustering-based AGentic memory framework where an SLM agent actively organizes memory by clustering. CLAG employs an SLM-driven router to assign incoming memories to semantically coherent clusters and autonomously generates cluster-specific profiles, including topic summaries and descriptive tags, to establish each cluster as a self-contained functional unit. By performing localized evolution within these structured neighborhoods, CLAG effectively reduces cross-topic interference and enhances internal memory density. During retrieval, the framework utilizes a two-stage process that first filters relevant clusters via their profiles, thereby excluding distractors and reducing the search space. Experiments on multiple QA datasets with three SLM backbones show that CLAG consistently improves answer quality and robustness over prior memory systems for agents, remaining lightweight and efficient.