SciMRC: Multi-perspective Scientific Machine Reading Comprehension

TL;DR

SciMRC introduces a multi-perspective dataset for scientific machine reading comprehension, capturing diverse understanding levels from beginners to experts, highlighting the importance of perspective-aware comprehension models.

Contribution

The paper presents SciMRC, a novel dataset with multiple perspectives, addressing the lack of multi-view understanding in existing scientific MRC datasets.

Findings

Pre-trained models benefit from multi-perspective data.

SciMRC is challenging for current models.

Different perspectives influence comprehension difficulty.

Abstract

Scientific machine reading comprehension (SMRC) aims to understand scientific texts through interactions with humans by given questions. As far as we know, there is only one dataset focused on exploring full-text scientific machine reading comprehension. However, the dataset has ignored the fact that different readers may have different levels of understanding of the text, and only includes single-perspective question-answer pairs, leading to a lack of consideration of different perspectives. To tackle the above problem, we propose a novel multi-perspective SMRC dataset, called SciMRC, which includes perspectives from beginners, students and experts. Our proposed SciMRC is constructed from 741 scientific papers and 6,057 question-answer pairs. Each perspective of beginners, students and experts contains 3,306, 1,800 and 951 QA pairs, respectively. The extensive experiments on SciMRC by utilizing pre-trained models suggest the importance of considering perspectives of SMRC, and demonstrate its challenging nature for machine comprehension.