Improving Coherence and Persistence in Agentic AI for System Optimization

TL;DR

Engram is an innovative agentic AI architecture that enhances system optimization by decoupling exploration from context limitations, enabling iterative learning and knowledge accumulation for complex problems.

Contribution

The paper introduces Engram, a novel agentic framework that separates long-horizon exploration from context constraints, improving system optimization across multiple domains.

Findings

Engram outperforms existing methods in diverse system optimization tasks.

It effectively accumulates knowledge across independent runs.

Demonstrates superior performance in multi-cloud multicast, LLM inference routing, and database cache optimization.

Abstract

Designing high-performance system heuristics is a creative, iterative process requiring experts to form hypotheses and execute multi-step conceptual shifts. While Large Language Models (LLMs) show promise in automating this loop, they struggle with complex system problems due to two critical failure modes: evolutionary neighborhood bias and the coherence ceiling. Evolutionary methods often remain trapped in local optima by relying on scalar benchmark scores, failing when coordinated multi-step changes are required. Conversely, existing agentic frameworks suffer from context degradation over long horizons or fail to accumulate knowledge across independent runs. We present Engram, an agentic researcher architecture that addresses these limitations by decoupling long-horizon exploration from the constraints of a single context window. Engram organizes exploration into a sequence of agents that iteratively design, test, and analyze mechanisms. At the conclusion of each run, an agent stores code snapshots, logs, and results in a persistent Archive and distills high-level modeling insights into a compact, persistent Research Digest. Subsequent agents then begin with a fresh context window, reading the Research Digest to build on prior discoveries. We find that Engram exhibits superior performance across diverse domains including multi-cloud multicast, LLM inference request routing, and optimizing KV cache reuse in databases with natural language queries.