Inter-Passage Verification for Multi-evidence Multi-answer QA

TL;DR

This paper introduces RI$^2$VER, a novel multi-answer QA framework that retrieves and individually verifies evidence passages through inter-passage synthesis, significantly improving accuracy on multi-evidence questions.

Contribution

It proposes a new retrieval-augmented QA framework with an inter-passage verification pipeline, enhancing multi-evidence answer accuracy beyond existing methods.

Findings

Achieves 11.17% higher F1 score on QAMPARI and RoMQA datasets.

Effectively handles questions requiring multi-evidence synthesis.

Outperforms baselines across various model sizes.

Abstract

Multi-answer question answering (QA), where questions can have many valid answers, presents a significant challenge for existing retrieval-augmented generation-based QA systems, as these systems struggle to retrieve and then synthesize a large number of evidence passages. To tackle these challenges, we propose a new multi-answer QA framework -- Retrieval-augmented Independent Reading with Inter-passage Verification (RI$^2$VER). Our framework retrieves a large set of passages and processes each passage individually to generate an initial high-recall but noisy answer set. Then we propose a new inter-passage verification pipeline that validates every candidate answer through (1) Verification Question Generation, (2) Gathering Additional Evidence, and (3) Verification with inter-passage synthesis. Evaluations on the QAMPARI and RoMQA datasets demonstrate that our framework significantly outperforms existing baselines across various model sizes, achieving an average F1 score improvement of 11.17%. Further analysis validates that our inter-passage verification pipeline enables our framework to be particularly beneficial for questions requiring multi-evidence synthesis.