When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges

TL;DR

This paper reveals that KV cache reuse in multi-agent LLM systems can disrupt judge consistency, especially in later candidate blocks, necessitating dedicated design for judge-centric inference.

Contribution

It uncovers a failure mode of KV cache reuse affecting judge behavior and emphasizes the importance of explicit cross-candidate interaction for reliable judge inference.

Findings

KV cache reuse can weaken cross-candidate attention in judges

Judge consistency is compromised by reuse strategies, especially in later candidates

Explicit cross-candidate interaction is essential for preserving decision accuracy

Abstract

Multi-agent LLM systems routinely generate multiple candidate responses that are aggregated by an LLM judge. To reduce the dominant prefill cost in such pipelines, recent work advocates KV cache reuse across partially shared contexts and reports substantial speedups for generation agents. In this work, we show that these efficiency gains do not transfer uniformly to judge-centric inference. Across GSM8K, MMLU, and HumanEval, we find that reuse strategies that are effective for execution agents can severely perturb judge behavior: end-task accuracy may appear stable, yet the judge's selection becomes highly inconsistent with dense prefill. We quantify this risk using Judge Consistency Rate (JCR) and provide diagnostics showing that reuse systematically weakens cross-candidate attention, especially for later candidate blocks. Our ablation further demonstrates that explicit cross-candidate interaction is crucial for preserving dense-prefill decisions. Overall, our results identify a previously overlooked failure mode of KV cache reuse and highlight judge-centric inference as a distinct regime that demands dedicated, risk-aware system design.