Context Modeling with Evidence Filter for Multiple Choice Question Answering

TL;DR

This paper introduces an evidence filtering method for multiple-choice question answering that models context relationships to highlight relevant evidence and filter out unrelated sentences, improving performance and interpretability.

Contribution

The paper presents a novel evidence filtering approach that reduces human effort and enhances evidence extraction in MCQA tasks, outperforming existing models on OpenbookQA.

Findings

Outperforms models with the same backbone and more data on OpenbookQA

Effectively highlights evidence sentences and filters unrelated ones

Demonstrates interpretability through parameter analysis

Abstract

Multiple-Choice Question Answering (MCQA) is a challenging task in machine reading comprehension. The main challenge in MCQA is to extract "evidence" from the given context that supports the correct answer. In the OpenbookQA dataset, the requirement of extracting "evidence" is particularly important due to the mutual independence of sentences in the context. Existing work tackles this problem by annotated evidence or distant supervision with rules which overly rely on human efforts. To address the challenge, we propose a simple yet effective approach termed evidence filtering to model the relationships between the encoded contexts with respect to different options collectively and to potentially highlight the evidence sentences and filter out unrelated sentences. In addition to the effective reduction of human efforts of our approach compared, through extensive experiments on OpenbookQA, we show that the proposed approach outperforms the models that use the same backbone and more training data; and our parameter analysis also demonstrates the interpretability of our approach.