A Span-Extraction Dataset for Chinese Machine Reading Comprehension

TL;DR

This paper introduces a new Chinese span-extraction dataset for machine reading comprehension, addressing the lack of Chinese datasets and providing resources to advance research in this language.

Contribution

It presents a large-scale Chinese MRC dataset with real questions and a challenge set, along with baseline systems and hosting a dedicated evaluation workshop.

Findings

Baseline systems demonstrate the dataset's difficulty.

The challenge set requires multi-sentence inference.

Resources are publicly available for research advancement.

Abstract

Machine Reading Comprehension (MRC) has become enormously popular recently and has attracted a lot of attention. However, the existing reading comprehension datasets are mostly in English. In this paper, we introduce a Span-Extraction dataset for Chinese machine reading comprehension to add language diversities in this area. The dataset is composed by near 20,000 real questions annotated on Wikipedia paragraphs by human experts. We also annotated a challenge set which contains the questions that need comprehensive understanding and multi-sentence inference throughout the context. We present several baseline systems as well as anonymous submissions for demonstrating the difficulties in this dataset. With the release of the dataset, we hosted the Second Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC 2018). We hope the release of the dataset could further accelerate the Chinese machine reading comprehension research. Resources are available: https://github.com/ymcui/cmrc2018