Q-KVComm: Efficient Multi-Agent Communication Via Adaptive KV Cache Compression

TL;DR

Q-KVComm introduces an adaptive, compressed KV cache protocol for multi-agent LLM systems, significantly reducing bandwidth while preserving semantic integrity across diverse tasks and models.

Contribution

It presents a novel adaptive quantization and hybrid extraction method for direct KV cache transmission, enabling efficient multi-agent communication.

Findings

Achieves 5-6x compression ratios with high semantic fidelity.

Maintains coherence scores above 0.77 across datasets.

Works effectively across models from 1.1B to 1.5B parameters.

Abstract

Multi-agent Large Language Model (LLM) systems face a critical bottleneck: redundant transmission of contextual information between agents consumes excessive bandwidth and computational resources. Traditional approaches discard internal semantic representations and transmit raw text, forcing receiving agents to recompute similar representations from scratch. We introduce Q-KVComm, a new protocol that enables direct transmission of compressed key-value (KV) cache representations between LLM agents. Q-KVComm combines three key innovations: (1) adaptive layer-wise quantization that allocates variable bit-widths based on sensitivity profiling, (2) hybrid information extraction that preserves critical facts across content domains, and (3) heterogeneous model calibration establishing cross-architecture communication. Extensive experiments across three diverse question-answering datasets demonstrate that Q-KVComm achieves 5-6x compression ratios while maintaining semantic fidelity, with coherence quality scores above 0.77 across all scenarios. The protocol exhibits robust performance across model sizes (1.1B-1.5B parameters) and adapts to real-world applications including conversational QA and multi-hop reasoning. Our work establishes a new paradigm for LLM agent communication, shifting from text-based to representation-based information exchange.