BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planning

TL;DR

BrainMem introduces a human-inspired, training-free hierarchical memory system for embodied agents, significantly improving long-horizon task success in complex 3D environments by enabling memory-driven reasoning and adaptation.

Contribution

It presents BrainMem, a novel evolving memory system that integrates seamlessly with LLMs, enhancing embodied task planning without additional training or fine-tuning.

Findings

BrainMem improves success rates on four benchmark tasks.

It enhances performance on long-horizon and spatially complex tasks.

The system reduces reliance on prompt engineering.

Abstract

Embodied task planning requires agents to execute long-horizon, goal-directed actions in complex 3D environments, where success depends on both immediate perception and accumulated experience across tasks. However, most existing LLM-based planners are stateless and reactive, operating without persistent memory and therefore repeating errors and struggling with spatial or temporal dependencies. We propose BrainMem(Brain-Inspired Evolving Memory), a training-free hierarchical memory system that equips embodied agents with working, episodic, and semantic memory inspired by human cognition. BrainMem continuously transforms interaction histories into structured knowledge graphs and distilled symbolic guidelines, enabling planners to retrieve, reason over, and adapt behaviors from past experience without any model fine-tuning or additional training. This plug-and-play design integrates seamlessly with arbitrary multi-modal LLMs and greatly reduces reliance on task-specific prompt engineering. Extensive experiments on four representative benchmarks, including EB-ALFRED, EB-Navigation, EB-Manipulation, and EB-Habitat, demonstrate that BrainMem significantly enhances task success rates across diverse models and difficulty subsets, with the largest gains observed on long-horizon and spatially complex tasks. These results highlight evolving memory as a promising and scalable mechanism for generalizable embodied intelligence.