Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answering

TL;DR

The paper introduces the DIVA framework, which enhances retrieval-augmented question answering by diversifying, verifying, and adapting retrieved passages to improve accuracy, robustness, and efficiency in handling ambiguous queries.

Contribution

DIVA is a novel framework that improves upon existing methods by diversifying retrieval, verifying passage quality, and adapting responses, addressing low-quality retrieval issues efficiently.

Findings

Improves QA accuracy and robustness.

Handles low-quality retrieval effectively.

Enhances efficiency over iterative approaches.

Abstract

The retrieval augmented generation (RAG) framework addresses an ambiguity in user queries in QA systems by retrieving passages that cover all plausible interpretations and generating comprehensive responses based on the passages. However, our preliminary studies reveal that a single retrieval process often suffers from low quality results, as the retrieved passages frequently fail to capture all plausible interpretations. Although the iterative RAG approach has been proposed to address this problem, it comes at the cost of significantly reduced efficiency. To address these issues, we propose the diversify-verify-adapt (DIVA) framework. DIVA first diversifies the retrieved passages to encompass diverse interpretations. Subsequently, DIVA verifies the quality of the passages and adapts the most suitable approach tailored to their quality. This approach improves the QA systems accuracy and robustness by handling low quality retrieval issue in ambiguous questions, while enhancing efficiency.