CompAct: Compressing Retrieved Documents Actively for Question Answering

TL;DR

CompAct is a new framework that actively compresses retrieved documents to improve question answering performance, achieving high compression rates and flexibility as a plug-in module for retrieval-augmented systems.

Contribution

It introduces an active document compression method that effectively condenses extensive information without losing key details, enhancing multi-hop question answering.

Findings

Significant performance improvements on multi-hop QA benchmarks.

Achieves up to 47x compression rate.

Operates as a flexible, cost-efficient plug-in module.

Abstract

Retrieval-augmented generation supports language models to strengthen their factual groundings by providing external contexts. However, language models often face challenges when given extensive information, diminishing their effectiveness in solving questions. Context compression tackles this issue by filtering out irrelevant information, but current methods still struggle in realistic scenarios where crucial information cannot be captured with a single-step approach. To overcome this limitation, we introduce CompAct, a novel framework that employs an active strategy to condense extensive documents without losing key information. Our experiments demonstrate that CompAct brings significant improvements in both performance and compression rate on multi-hop question-answering benchmarks. CompAct flexibly operates as a cost-efficient plug-in module with various off-the-shelf retrievers or readers, achieving exceptionally high compression rates (47x).