QAEA-DR: A Unified Text Augmentation Framework for Dense Retrieval

TL;DR

This paper introduces QAEA-DR, a text augmentation framework that enhances dense retrieval by transforming raw texts into information-rich formats using large language models, improving query-text matching without altering existing retrieval methods.

Contribution

The paper proposes a novel augmentation framework combining question-answer pairs and event extraction, with a scoring mechanism to improve dense retrieval performance.

Findings

Improved retrieval accuracy demonstrated in experiments.

Effective augmentation method without changing existing models.

Theoretical analysis supports the approach's validity.

Abstract

In dense retrieval, embedding long texts into dense vectors can result in information loss, leading to inaccurate query-text matching. Additionally, low-quality texts with excessive noise or sparse key information are unlikely to align well with relevant queries. Recent studies mainly focus on improving the sentence embedding model or retrieval process. In this work, we introduce a novel text augmentation framework for dense retrieval. This framework transforms raw documents into information-dense text formats, which supplement the original texts to effectively address the aforementioned issues without modifying embedding or retrieval methodologies. Two text representations are generated via large language models (LLMs) zero-shot prompting: question-answer pairs and element-driven events. We term this approach QAEA-DR: unifying question-answer generation and event extraction in a text augmentation framework for dense retrieval. To further enhance the quality of generated texts, a scoring-based evaluation and regeneration mechanism is introduced in LLM prompting. Our QAEA-DR model has a positive impact on dense retrieval, supported by both theoretical analysis and empirical experiments.