RPMS: Enhancing LLM-Based Embodied Planning through Rule-Augmented Memory Synergy

TL;DR

RPMS improves LLM-based embodied planning by integrating rule retrieval and memory management, significantly boosting success rates in complex tasks and demonstrating the importance of structured rule-based grounding and conditional memory use.

Contribution

The paper introduces RPMS, a novel architecture that combines rule-augmented memory retrieval with conflict arbitration to enhance LLM agent performance in embodied environments.

Findings

RPMS achieves 59.7% success on ALFWorld tasks, outperforming baselines.

Rule retrieval alone contributes +14.9 percentage points to success.

Conditional use of episodic memory improves performance across environments.

Abstract

LLM agents often fail in closed-world embodied environments because actions must satisfy strict preconditions -- such as location, inventory, and container states -- and failure feedback is sparse. We identify two structurally coupled failure modes: (P1) invalid action generation and (P2) state drift, each amplifying the other in a degenerative cycle. We present RPMS, a conflict-managed architecture that enforces action feasibility via structured rule retrieval, gates memory applicability via a lightweight belief state, and resolves conflicts between the two sources via rules-first arbitration. On ALFWorld (134 unseen tasks), RPMS achieves 59.7% single-trial success with Llama 3.1 8B (+23.9 pp over baseline) and 98.5% with Claude Sonnet 4.5 (+11.9 pp); of the 8B gain, rule retrieval alone contributes +14.9 pp (statistically significant), making it the dominant factor. A key finding is that episodic memory is conditionally useful: it harms performance on some task types when used without grounding, but becomes a stable net positive once filtered by current state and constrained by explicit action rules. Adapting RPMS to ScienceWorld with GPT-4 yields consistent gains across all ablation conditions (avg. score 54.0 vs. 44.9 for the ReAct baseline), providing transfer evidence that the core mechanisms hold across structurally distinct environments.