From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition

TL;DR

This paper introduces EDU-based Context Compressor, a novel method that preserves document structure during context compression for LLMs, improving accuracy and efficiency in long-document tasks.

Contribution

The paper proposes a structure-then-select framework using Elementary Discourse Units and introduces StructBench for evaluation, advancing structured context compression techniques.

Findings

Achieves state-of-the-art structural prediction accuracy

Outperforms frontier LLMs in cost and performance

Enhances downstream task performance with structure-aware compression

Abstract

Managing extensive context remains a critical bottleneck for Large Language Models (LLMs), particularly in applications like long-document question answering and autonomous agents where lengthy inputs incur high computational costs and introduce noise. Existing compression techniques often disrupt local coherence through discrete token removal or rely on implicit latent encoding that suffers from positional bias and incompatibility with closed-source APIs. To address these limitations, we introduce the EDU-based Context Compressor, a novel explicit compression framework designed to preserve both global structure and fine-grained details. Our approach reformulates context compression as a structure-then-select process. First, our LingoEDU transforms linear text into a structural relation tree of Elementary Discourse Units (EDUs) which are anchored strictly to source indices to eliminate hallucination. Second, a lightweight ranking module selects query-relevant sub-trees for linearization. To rigorously evaluate structural understanding, we release StructBench, a manually annotated dataset of 248 diverse documents. Empirical results demonstrate that our method achieves state-of-the-art structural prediction accuracy and significantly outperforms frontier LLMs while reducing costs. Furthermore, our structure-aware compression substantially enhances performance across downstream tasks ranging from long-context tasks to complex Deep Search scenarios.