EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation

TL;DR

EXIT is a novel extractive compression framework that improves retrieval-augmented question answering by adaptively selecting context sentences, enhancing accuracy and reducing latency compared to existing methods.

Contribution

EXIT introduces a context-aware, adaptive sentence classification method that preserves dependencies, outperforming existing extractive and abstractive compression techniques in QA tasks.

Findings

Outperforms existing compression methods in QA accuracy

Reduces inference time and token count significantly

Effective on both single-hop and multi-hop QA tasks

Abstract

We introduce EXIT, an extractive context compression framework that enhances both the effectiveness and efficiency of retrieval-augmented generation (RAG) in question answering (QA). Current RAG systems often struggle when retrieval models fail to rank the most relevant documents, leading to the inclusion of more context at the expense of latency and accuracy. While abstractive compression methods can drastically reduce token counts, their token-by-token generation process significantly increases end-to-end latency. Conversely, existing extractive methods reduce latency but rely on independent, non-adaptive sentence selection, failing to fully utilize contextual information. EXIT addresses these limitations by classifying sentences from retrieved documents - while preserving their contextual dependencies - enabling parallelizable, context-aware extraction that adapts to query complexity and retrieval quality. Our evaluations on both single-hop and multi-hop QA tasks show that EXIT consistently surpasses existing compression methods and even uncompressed baselines in QA accuracy, while also delivering substantial reductions in inference time and token count. By improving both effectiveness and efficiency, EXIT provides a promising direction for developing scalable, high-quality QA solutions in RAG pipelines. Our code is available at https://github.com/ThisIsHwang/EXIT