Choosing How to Remember: Adaptive Memory Structures for LLM Agents

TL;DR

This paper introduces FluxMem, a unified framework for adaptive memory organization in LLM agents, enabling context-aware memory structure selection and fusion to improve long-horizon interaction performance.

Contribution

FluxMem provides a novel adaptive memory system with multiple structures and a probabilistic gating mechanism, addressing limitations of fixed memory approaches in LLM agents.

Findings

Achieves 9.18% and 6.14% improvements on benchmarks.

Learns to select memory structures based on interaction features.

Replaces brittle similarity thresholds with probabilistic memory fusion.

Abstract

Memory is critical for enabling large language model (LLM) based agents to maintain coherent behavior over long-horizon interactions. However, existing agent memory systems suffer from two key gaps: they rely on a one-size-fits-all memory structure and do not model memory structure selection as a context-adaptive decision, limiting their ability to handle heterogeneous interaction patterns and resulting in suboptimal performance. We propose a unified framework, FluxMem, that enables adaptive memory organization for LLM agents. Our framework equips agents with multiple complementary memory structures. It explicitly learns to select among these structures based on interaction-level features, using offline supervision derived from downstream response quality and memory utilization. To support robust long-horizon memory evolution, we further introduce a three-level memory hierarchy and a Beta Mixture Model-based probabilistic gate for distribution-aware memory fusion, replacing brittle similarity thresholds. Experiments on two long-horizon benchmarks, PERSONAMEM and LoCoMo, demonstrate that our method achieves average improvements of 9.18% and 6.14%.