An Understanding-Oriented Robust Machine Reading Comprehension Model

TL;DR

This paper introduces a novel understanding-oriented machine reading comprehension model that enhances robustness against sensitivity, stability, and generalization issues through integrated modules and multi-task learning, outperforming existing models on benchmark datasets.

Contribution

The paper presents a new robust MRC model with a natural language inference module, memory-guided multi-head attention, and multilanguage learning, addressing key robustness challenges.

Findings

Achieves superior performance on robustness benchmarks

Effectively addresses over sensitivity and over stability

Improves generalization across multiple languages

Abstract

Although existing machine reading comprehension models are making rapid progress on many datasets, they are far from robust. In this paper, we propose an understanding-oriented machine reading comprehension model to address three kinds of robustness issues, which are over sensitivity, over stability and generalization. Specifically, we first use a natural language inference module to help the model understand the accurate semantic meanings of input questions so as to address the issues of over sensitivity and over stability. Then in the machine reading comprehension module, we propose a memory-guided multi-head attention method that can further well understand the semantic meanings of input questions and passages. Third, we propose a multilanguage learning mechanism to address the issue of generalization. Finally, these modules are integrated with a multi-task learning based method. We evaluate our model on three benchmark datasets that are designed to measure models robustness, including DuReader (robust) and two SQuAD-related datasets. Extensive experiments show that our model can well address the mentioned three kinds of robustness issues. And it achieves much better results than the compared state-of-the-art models on all these datasets under different evaluation metrics, even under some extreme and unfair evaluations. The source code of our work is available at: https://github.com/neukg/RobustMRC.