A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
Danqi Chen, Jason Bolton, Christopher D. Manning

TL;DR
This paper thoroughly examines the CNN/Daily Mail reading comprehension task, analyzing the depth of language understanding needed and demonstrating that simple systems can achieve near-peak performance, surpassing previous state-of-the-art results.
Contribution
It provides a detailed analysis of the task's complexity and shows that straightforward models can attain high accuracy, indicating the task may be nearing its performance ceiling.
Findings
Simple systems achieved 73.6% and 76.6% accuracy.
Analysis suggests the task may be approaching its performance ceiling.
Hand-analysis reveals the level of language understanding required.
Abstract
Enabling a computer to understand a document so that it can answer comprehension questions is a central, yet unsolved goal of NLP. A key factor impeding its solution by machine learned systems is the limited availability of human-annotated data. Hermann et al. (2015) seek to solve this problem by creating over a million training examples by pairing CNN and Daily Mail news articles with their summarized bullet points, and show that a neural network can then be trained to give good performance on this task. In this paper, we conduct a thorough examination of this new reading comprehension task. Our primary aim is to understand what depth of language understanding is required to do well on this task. We approach this from one side by doing a careful hand-analysis of a small subset of the problems and from the other by showing that simple, carefully designed systems can obtain accuracies of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Text Readability and Simplification
