Fix the Structural Bottleneck: Context Compression via Explicit Information Transmission

TL;DR

This paper introduces ComprExIT, a novel context compression framework for long-context LLMs that enhances information preservation and efficiency through explicit transmission and adaptive feature selection.

Contribution

It identifies key bottlenecks in existing compression methods and proposes a new framework that significantly improves performance with minimal additional training.

Findings

ComprExIT outperforms strong baselines on 12 datasets.

Achieves up to 18.5% F1 improvement.

Provides over 2x faster compression with minimal trainable parameters.

Abstract

Long-context LLM agents often struggle with growing token, memory, and latency costs, making efficient context compression essential for practical deployment. Existing LLM-as-a-compressor methods remain noticeably inferior to using the full context. We find that this gap partly stems from their inability to preserve contextual information effectively. In this work, we revisit context compression from a structural perspective and identify two key bottlenecks in standard LLM-based compressors: limited coordination among compression tokens during information aggregation, and layerwise dilution that weakens useful signals from intermediate hidden states. To address these limitations, we propose ComprExIT, a new context compression framework based on explicit information transmission. ComprExIT adaptively selects features across frozen LLM layers, then allocates information from anchors to compression slots through a globally coordinated transport plan. Experiments on 12 datasets show that ComprExIT consistently outperforms strong soft-compression baselines, improving average F1 by up to 18.5%, while adding only ~1% trainable parameters and achieving more than 2x faster compression than the fastest baselines. The code will be released upon acceptance.