R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memory

TL;DR

R2D2 introduces a memory and reflection-based framework for web agents that significantly improves navigation accuracy and task success rates by dynamically reconstructing environments and learning from past errors.

Contribution

The paper presents R2D2, a novel framework combining Remember and Reflect paradigms to enhance web agent navigation and decision-making capabilities.

Findings

50% reduction in navigation errors

Threefold increase in task completion rates

Effective environment reconstruction and error learning

Abstract

The proliferation of web agents necessitates advanced navigation and interaction strategies within complex web environments. Current models often struggle with efficient navigation and action execution due to limited visibility and understanding of web structures. Our proposed R2D2 framework addresses these challenges by integrating two paradigms: Remember and Reflect. The Remember paradigm uses a replay buffer that aids agents in reconstructing the web environment dynamically, thus enabling the formulation of a detailed "map" of previously visited pages. This helps in reducing navigational errors and optimizing the decision-making process during web interactions. Conversely, the Reflect paradigm allows agents to learn from past mistakes by providing a mechanism for error analysis and strategy refinement, enhancing overall task performance. We evaluate R2D2 using the WebArena benchmark, demonstrating substantial improvements over existing methods, including a 50% reduction in navigation errors and a threefold increase in task completion rates. Our findings suggest that a combination of memory-enhanced navigation and reflective learning promisingly advances the capabilities of web agents, potentially benefiting various applications such as automated customer service and personal digital assistants.