AgentRM: An OS-Inspired Resource Manager for LLM Agent Systems

TL;DR

This paper introduces AgentRM, an OS-inspired resource management system for LLM agent frameworks, significantly improving responsiveness, resource utilization, and information retention by addressing scheduling and memory challenges.

Contribution

AgentRM applies OS principles to LLM agent resource management, introducing a scheduler and context manager that enhance performance and retention.

Findings

P95 latency reduced by 86% with AgentRM-MLFQ

Zombie agents eliminated, throughput increased by 168%

Key information retention improved to 100% with AgentRM-CLM

Abstract

Large Language Model (LLM) agent systems have experienced rapid adoption across diverse domains, yet they suffer from critical user experience problems that limit their practical deployment. Through an empirical analysis of over 40,000 GitHub issues from six major agent frameworks (OpenClaw, AutoGen, CrewAI, LangGraph, Codex, Claude Code), we identify two fundamental resource management challenges: (1) scheduling failures leading to system unresponsiveness due to blocking, zombie processes, and rate limit cascades, and (2) context degradation causing agent "amnesia" from unbounded memory growth and poor retention policies. Drawing inspiration from decades of operating systems research, we present AgentRM, a middleware resource manager that treats agent resources analogously to OS resources. AgentRM employs a Multi-Level Feedback Queue (MLFQ) scheduler with zombie reaping and rate-limit-aware admission control, coupled with a three-tier Context Lifecycle Manager that implements adaptive compaction and hibernation mechanisms. Our evaluation demonstrates significant improvements: AgentRM-MLFQ reduces P95 latency by 86%, decreases lane waste by 96%, and increases throughput by 168% while eliminating zombie agents (0 vs. 29 baseline). AgentRM-CLM achieves 100% key information retention with 95% quality score compared to 65.1% retention and 87% quality for existing approaches, albeit with higher compaction costs (34,330 vs. 17,212 tokens).