Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents

TL;DR

This paper introduces Contextual Memory Virtualisation (CMV), a DAG-based system for managing and structurally losslessly trimming LLM session states, enabling efficient context reuse and significant token reduction.

Contribution

It presents a novel DAG-based session state management system with a lossless trimming algorithm that preserves user interactions while reducing token usage in LLM sessions.

Findings

Trimming reduces token count by an average of 20%, up to 86%.

CMV enables context reuse across parallel sessions.

Economic viability demonstrated in real-world coding sessions.

Abstract

As large language models engage in extended reasoning tasks, they accumulate significant state -- architectural mappings, trade-off decisions, codebase conventions -- within the context window. This understanding is lost when sessions reach context limits and undergo lossy compaction. We propose Contextual Memory Virtualisation (CMV), a system that treats accumulated LLM understanding as version-controlled state. Borrowing from operating system virtual memory, CMV models session history as a Directed Acyclic Graph (DAG) with formally defined snapshot, branch, and trim primitives that enable context reuse across independent parallel sessions. We introduce a three-pass structurally lossless trimming algorithm that preserves every user message and assistant response verbatim while reducing token counts by a mean of 20% and up to 86% for sessions with significant overhead by stripping mechanical bloat such as raw tool outputs, base64 images, and metadata. A single-user case-study evaluation across 76 real-world coding sessions demonstrates that trimming remains economically viable under prompt caching, with the strongest gains in mixed tool-use sessions, which average 39% reduction and reach break-even within 10 turns. A reference implementation is available at https://github.com/CosmoNaught/claude-code-cmv.