End-to-End Beam Retrieval for Multi-Hop Question Answering

TL;DR

This paper introduces Beam Retrieval, an end-to-end multi-hop passage retrieval framework that jointly optimizes retrieval across all hops, significantly improving performance on complex multi-hop question answering datasets.

Contribution

The paper proposes Beam Retrieval, a novel end-to-end approach that models multi-hop retrieval with multiple hypotheses, enhancing retrieval accuracy and system performance in multi-hop QA tasks.

Findings

Achieves nearly 50% improvement on MuSiQue-Ans

Surpasses all previous retrievers on HotpotQA

Reaches 99.9% precision on 2WikiMultiHopQA

Abstract

Multi-hop question answering (QA) involves finding multiple relevant passages and step-by-step reasoning to answer complex questions, indicating a retrieve-and-read paradigm. However, previous retrievers were customized for two-hop questions, and most of them were trained separately across different hops, resulting in a lack of supervision over the entire multi-hop retrieval process and leading to poor performance in complicated scenarios beyond two hops. In this work, we introduce Beam Retrieval, an end-to-end beam retrieval framework for multi-hop QA. This approach models the multi-hop retrieval process in an end-to-end manner by jointly optimizing an encoder and two classification heads across all hops. Moreover, Beam Retrieval maintains multiple partial hypotheses of relevant passages at each step, expanding the search space and reducing the risk of missing relevant passages. To establish a complete QA system, we incorporate a supervised reader or a large language model (LLM). Experimental results demonstrate that Beam Retrieval achieves a nearly 50% improvement compared with baselines on challenging MuSiQue-Ans, and it also surpasses all previous retrievers on HotpotQA and achieves 99.9% precision on 2WikiMultiHopQA. Providing high-quality context, Beam Retrieval helps our supervised reader achieve new state-of-the-art performance and substantially improves the few-shot QA performance of LLMs.