ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents

TL;DR

This paper introduces BACM, a reinforcement learning-based method for managing limited context in long-horizon search agents, improving performance across various tasks and budgets.

Contribution

It proposes BACM-RL, an end-to-end reinforcement learning approach for adaptive context compression under budget constraints, outperforming prior methods.

Findings

BACM-RL achieves over 1.6x gains in high-complexity tasks.

It maintains strong performance as context budgets decrease.

Experiments cover compositional multi-objective QA and web browsing benchmarks.

Abstract

LLM-based agents show strong potential for long-horizon reasoning, yet their context size is limited by deployment factors (e.g., memory, latency, and cost), yielding a constrained context budget. As interaction histories grow, this induces a trade-off between retaining past information and staying within the context limit. To address this challenge, we propose Budget-Aware Context Management (BACM), which formulates context management as a sequential decision problem with a context budget constraint. It enables agents to assess the available budget before incorporating new observations and decide when and how much of the interaction history to compress. We further develop BACM-RL, an end-to-end curriculum-based reinforcement learning approach that learns compression strategies under varying context budgets. Experiments on compositional multi-objective QA and long-horizon web browsing benchmarks show that BACM-RL consistently outperforms prior methods across model scales and task complexities, achieving over $1.6\times$ gains over strong baselines in high-complexity settings, while maintaining strong advantages as budgets shrink, where most methods exhibit a downward performance trend.