ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents

TL;DR

ContextWeaver introduces a dependency-structured memory system for LLM agents, enhancing long-context reasoning by organizing interactions into a graph and selectively retrieving relevant information.

Contribution

It presents a novel memory framework that models logical dependencies and summarizes reasoning paths, improving performance and efficiency in LLM agent interactions.

Findings

Outperforms sliding-window baseline in pass@1 on SWE-Bench benchmarks.

Reduces reasoning steps and token usage.

Supports dependency-based construction, summarization, and validation.

Abstract

Large language model (LLM) agents often struggle in long-context interactions. As the agent accumulates more interaction history, context management approaches such as sliding window and prompt compression may omit earlier structured information that later steps rely on. Recent retrieval-based memory systems surface relevant content but still overlook the causal and logical structure needed for multi-step reasoning. We introduce ContextWeaver, a selective and dependency-structured memory framework that organizes an agent's interaction trace into a graph of reasoning steps and selects the relevant context for future actions. Unlike prior context management approaches, ContextWeaver supports: (1) dependency-based construction and traversal that link each step to the earlier steps it relies on; (2) compact dependency summarization that condenses root-to-step reasoning paths into reusable units; and (3) a lightweight validation layer that incorporates execution feedback. On the SWE-Bench Verified and Lite benchmarks, ContextWeaver improves performance over a sliding-window baseline in pass@1, while reducing reasoning steps and token usage. Our observations suggest that modeling logical dependencies provides a stable and scalable memory mechanism for LLM agents that use tools.