FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression

TL;DR

FAEDKV introduces a training-free, frequency-domain KV cache compression method for LLMs that preserves unbiased information from all tokens, improving long-context performance without retraining.

Contribution

The paper proposes FAEDKV, a novel Infinite-Window Fourier Transform-based framework for unbiased KV cache compression, addressing biases of existing methods without requiring retraining.

Findings

Outperforms existing methods by up to 22% on LongBench.

Achieves superior position-agnostic retrieval accuracy.

Effectively preserves early and recent context information.

Abstract

The efficacy of Large Language Models (LLMs) in long-context tasks is often hampered by the substantial memory footprint and computational demands of the Key-Value (KV) cache. Current compression strategies, including token eviction and learned projections, frequently lead to biased representations -- either by overemphasizing recent/high-attention tokens or by repeatedly degrading information from earlier context -- and may require costly model retraining. We present FAEDKV (Frequency-Adaptive Infinite-Window for KV cache), a novel, training-free KV cache compression framework that ensures unbiased information retention. FAEDKV operates by transforming the KV cache into the frequency domain using a proposed Infinite-Window Fourier Transform (IWDFT). This approach allows for the equalized contribution of all tokens to the compressed representation, effectively preserving both early and recent contextual information. A preliminary frequency ablation study identifies critical spectral components for layer-wise, targeted compression. Experiments on LongBench benchmark demonstrate FAEDKV's superiority over existing methods by up to 22\%. In addition, our method shows superior, position-agnostic retrieval accuracy on the Needle-In-A-Haystack task compared to compression based approaches.