No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension

TL;DR

This paper introduces Reflection Net, a model for document-level machine reading comprehension that effectively handles various answer types, including no-answer cases, achieving top leaderboard performance.

Contribution

It presents a novel Reflection Net model with a two-step training process to better identify no-answer and wrong-answer cases in complex MRC tasks.

Findings

Achieved top 1 on both long and short answer leaderboards.

F1 scores of 77.2 for long answers and 64.1 for short answers.

Effective handling of diverse answer types including no-answer cases.

Abstract

The Natural Questions (NQ) benchmark set brings new challenges to Machine Reading Comprehension: the answers are not only at different levels of granularity (long and short), but also of richer types (including no-answer, yes/no, single-span and multi-span). In this paper, we target at this challenge and handle all answer types systematically. In particular, we propose a novel approach called Reflection Net which leverages a two-step training procedure to identify the no-answer and wrong-answer cases. Extensive experiments are conducted to verify the effectiveness of our approach. At the time of paper writing (May.~20,~2020), our approach achieved the top 1 on both long and short answer leaderboard, with F1 scores of 77.2 and 64.1, respectively.