Reading Comprehension Ability Test-A Turing Test for Reading Comprehension
Yuan Miao, Gongqi Lin, Yidan Hu, Chunyan Miao

TL;DR
This paper introduces the Reading Comprehension Ability Test (CAT), a Turing-test-like framework to evaluate and compare machine and human reading comprehension across multiple ability levels.
Contribution
It proposes a novel assessment method, CAT, to measure and differentiate machine and human reading comprehension abilities at various levels.
Findings
CAT can distinguish between different levels of comprehension ability.
It provides a benchmark for evaluating AI reading understanding.
The test aligns with human comprehension levels.
Abstract
Reading comprehension is an important ability of human intelligence. Literacy and numeracy are two most essential foundation for people to succeed at study, at work and in life. Reading comprehension ability is a core component of literacy. In most of the education systems, developing reading comprehension ability is compulsory in the curriculum from year one to year 12. It is an indispensable ability in the dissemination of knowledge. With the emerging artificial intelligence, computers start to be able to read and understand like people in some context. They can even read better than human beings for some tasks, but have little clue in other tasks. It will be very beneficial if we can identify the levels of machine comprehension ability, which will direct us on the further improvement. Turing test is a well-known test of the difference between computer intelligence and human…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Teaching and Learning Programming · Intelligent Tutoring Systems and Adaptive Learning
