Towards Interpretable and Reliable Reading Comprehension: A Pipeline Model with Unanswerability Prediction

TL;DR

This paper introduces an interpretable reading comprehension pipeline model capable of predicting unanswerable questions, enhancing interpretability and reliability while maintaining competitive performance on multi-hop QA tasks.

Contribution

The study proposes a novel pipeline RC model with unanswerability prediction and an end-to-end training method, improving interpretability and reliability in multi-hop QA.

Findings

Outperforms non-interpretable models on modified HotpotQA dataset

Achieves comparable results to previous models despite interpretability constraints

Effectively detects unanswerable questions to improve answer reliability

Abstract

Multi-hop QA with annotated supporting facts, which is the task of reading comprehension (RC) considering the interpretability of the answer, has been extensively studied. In this study, we define an interpretable reading comprehension (IRC) model as a pipeline model with the capability of predicting unanswerable queries. The IRC model justifies the answer prediction by establishing consistency between the predicted supporting facts and the actual rationale for interpretability. The IRC model detects unanswerable questions, instead of outputting the answer forcibly based on the insufficient information, to ensure the reliability of the answer. We also propose an end-to-end training method for the pipeline RC model. To evaluate the interpretability and the reliability, we conducted the experiments considering unanswerability in a multi-hop question for a given passage. We show that our end-to-end trainable pipeline model outperformed a non-interpretable model on our modified HotpotQA dataset. Experimental results also show that the IRC model achieves comparable results to the previous non-interpretable models in spite of the trade-off between prediction performance and interpretability.