Empowering Working Memory for Large Language Model Agents

TL;DR

This paper proposes a novel architecture for large language models that incorporates a working memory system inspired by cognitive psychology to improve memory retention and contextual reasoning across interactions.

Contribution

It introduces a centralized Working Memory Hub and Episodic Buffer to enhance LLMs' memory capabilities, addressing limitations of traditional memory designs.

Findings

Enhanced memory retention across episodes

Improved contextual reasoning in complex tasks

Blueprint for future memory-augmented LLMs

Abstract

Large language models (LLMs) have achieved impressive linguistic capabilities. However, a key limitation persists in their lack of human-like memory faculties. LLMs exhibit constrained memory retention across sequential interactions, hindering complex reasoning. This paper explores the potential of applying cognitive psychology's working memory frameworks, to enhance LLM architecture. The limitations of traditional LLM memory designs are analyzed, including their isolation of distinct dialog episodes and lack of persistent memory links. To address this, an innovative model is proposed incorporating a centralized Working Memory Hub and Episodic Buffer access to retain memories across episodes. This architecture aims to provide greater continuity for nuanced contextual reasoning during intricate tasks and collaborative scenarios. While promising, further research is required into optimizing episodic memory encoding, storage, prioritization, retrieval, and security. Overall, this paper provides a strategic blueprint for developing LLM agents with more sophisticated, human-like memory capabilities, highlighting memory mechanisms as a vital frontier in artificial general intelligence.