Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memory

TL;DR

This paper introduces MCMA, a learnable meta-cognitive memory management method for LLM agents that improves generalization and transfer by hierarchically organizing and abstracting memory, decoupling memory management from task execution.

Contribution

It proposes a novel, learnable memory abstraction approach that enhances transferability and generalization in LLM agents by hierarchical memory organization and a dedicated memory copilot.

Findings

Significant performance improvements on ALFWorld, ScienceWorld, and BabyAI.

Enhanced out-of-distribution generalization and cross-task transfer.

Effective hierarchical memory abstraction and management.

Abstract

Large language model (LLM) agents increasingly rely on accumulated memory to solve long-horizon decision-making tasks. However, most existing approaches store memory in fixed representations and reuse it at a single or implicit level of abstraction, which limits generalization and often leads to negative transfer when distribution shift. This paper proposes the Meta-Cognitive Memory Abstraction method (MCMA), which treats memory abstraction as a learnable cognitive skill rather than a fixed design choice. MCMA decouples task execution from memory management by combining a frozen task model with a learned memory copilot. The memory copilot is trained using direct preference optimization, it determines how memories should be structured, abstracted, and reused. Memories are further organized into a hierarchy of abstraction levels, enabling selective reuse based on task similarity. When no memory is transferable, MCMA transfers the ability to abstract and manage memory by transferring the memory copilot. Experiments on ALFWorld, ScienceWorld, and BabyAI demonstrate substantial improvements in performance, out-of-distribution generalization, and cross-task transfer over several baselines.