Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehension

TL;DR

This paper introduces a multi-task learning approach for machine reading at scale that jointly trains information retrieval and reading comprehension, leading to state-of-the-art results on benchmark datasets.

Contribution

It proposes a simple, effective multi-task learning method that trains IR and RC together by considering answer spans, improving passage retrieval accuracy.

Findings

Achieved state-of-the-art performance on SQuAD with Wikipedia passages.

Significant improvements in IR accuracy through joint training.

Teaching models to identify answer spans enhances IR effectiveness.

Abstract

This study considers the task of machine reading at scale (MRS) wherein, given a question, a system first performs the information retrieval (IR) task of finding relevant passages in a knowledge source and then carries out the reading comprehension (RC) task of extracting an answer span from the passages. Previous MRS studies, in which the IR component was trained without considering answer spans, struggled to accurately find a small number of relevant passages from a large set of passages. In this paper, we propose a simple and effective approach that incorporates the IR and RC tasks by using supervised multi-task learning in order that the IR component can be trained by considering answer spans. Experimental results on the standard benchmark, answering SQuAD questions using the full Wikipedia as the knowledge source, showed that our model achieved state-of-the-art performance. Moreover, we thoroughly evaluated the individual contributions of our model components with our new Japanese dataset and SQuAD. The results showed significant improvements in the IR task and provided a new perspective on IR for RC: it is effective to teach which part of the passage answers the question rather than to give only a relevance score to the whole passage.