U3E: Unsupervised and Erasure-based Evidence Extraction for Machine Reading Comprehension

TL;DR

This paper introduces U3E, an unsupervised method for extracting evidence sentences in machine reading comprehension by simulating human memory decline through sentence erasure, improving evidence accuracy and model performance.

Contribution

The paper presents a novel unsupervised evidence extraction approach using sentence erasure and semantic understanding metrics, reducing reliance on labor-intensive annotations.

Findings

U3E outperforms typical methods in evidence extraction accuracy.

U3E significantly enhances overall model performance.

The approach is simple yet effective across datasets.

Abstract

More tasks in Machine Reading Comprehension(MRC) require, in addition to answer prediction, the extraction of evidence sentences that support the answer. However, the annotation of supporting evidence sentences is usually time-consuming and labor-intensive. In this paper, to address this issue and considering that most of the existing extraction methods are semi-supervised, we propose an unsupervised evidence extraction method (U3E). U3E takes the changes after sentence-level feature erasure in the document as input, simulating the decline in problem-solving ability caused by human memory decline. In order to make selections on the basis of fully understanding the semantics of the original text, we also propose metrics to quickly select the optimal memory model for this input changes. To compare U3E with typical evidence extraction methods and investigate its effectiveness in evidence extraction, we conduct experiments on different datasets. Experimental results show that U3E is simple but effective, not only extracting evidence more accurately, but also significantly improving model performance.