Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents

TL;DR

This paper introduces an interpretable multi-hop reading comprehension system that filters relevant documents, jointly predicts answers and supporting sentences, and achieves competitive results on HotpotQA.

Contribution

The paper presents a novel Select, Answer and Explain (SAE) system with a pairwise learning-to-rank document classifier and multi-task learning for interpretability and improved accuracy.

Findings

Achieves top competitive performance on HotpotQA in distractor setting

Effectively filters out irrelevant documents to reduce distraction

Jointly predicts answers and supporting sentences with attention-based interaction

Abstract

Interpretable multi-hop reading comprehension (RC) over multiple documents is a challenging problem because it demands reasoning over multiple information sources and explaining the answer prediction by providing supporting evidences. In this paper, we propose an effective and interpretable Select, Answer and Explain (SAE) system to solve the multi-document RC problem. Our system first filters out answer-unrelated documents and thus reduce the amount of distraction information. This is achieved by a document classifier trained with a novel pairwise learning-to-rank loss. The selected answer-related documents are then input to a model to jointly predict the answer and supporting sentences. The model is optimized with a multi-task learning objective on both token level for answer prediction and sentence level for supporting sentences prediction, together with an attention-based interaction between these two tasks. Evaluated on HotpotQA, a challenging multi-hop RC data set, the proposed SAE system achieves top competitive performance in distractor setting compared to other existing systems on the leaderboard.