Prompt Compression in Production Task Orchestration: A Pre-Registered Randomized Trial

TL;DR

This study evaluates prompt compression strategies in production multi-agent task orchestration, revealing that moderate compression reduces costs effectively, while aggressive compression can increase costs due to output expansion, emphasizing careful policy design.

Contribution

The paper provides the first empirical analysis of prompt compression effects on production costs and output quality in multi-agent orchestration, introducing structure-aware strategies and cost-similarity trade-offs.

Findings

Moderate compression (r=0.5) reduces total inference cost by 27.9%.

Aggressive compression (r=0.2) increases total cost by 1.8%.

Recency-weighted compression achieves 23.5% savings and balances cost and similarity.

Abstract

The economics of prompt compression depend not only on reducing input tokens but on how compression changes output length, which is typically priced several times higher. We evaluate this in a pre-registered six-arm randomized controlled trial of prompt compression on production multi-agent task-orchestration, analyzing 358 successful Claude Sonnet 4.5 runs (59-61 per arm) drawn from a randomized corpus of 1,199 real orchestration instructions. We compare an uncompressed control with three uniform retention rates (r=0.8, 0.5, 0.2) and two structure-aware strategies (entropy-adaptive and recency-weighted), measuring total inference cost (input+output) and embedding-based response similarity. Moderate compression (r=0.5) reduced mean total cost by 27.9%, while aggressive compression (r=0.2) increased mean cost by 1.8% despite substantial input reduction, consistent with small mean output expansion (1.03x vs. control) and heavy-tailed uncertainty. Recency-weighted compression achieved 23.5% savings and, together with moderate compression, occupied the empirical cost-similarity Pareto frontier, whereas aggressive compression was dominated on both cost and similarity. These results show that "compress more" is not a reliable production heuristic and that output tokens must be treated as a first-class outcome when designing compression policies.