Option Comparison Network for Multiple-choice Reading Comprehension

TL;DR

This paper introduces an option comparison network that compares answer choices at the word level to improve multiple-choice reading comprehension, outperforming existing models and surpassing human performance on the RACE dataset.

Contribution

The proposed model mimics human multi-granularity comparison of options, using word-level encoding and attention to enhance reasoning in MCRC tasks.

Findings

Outperforms existing models on RACE dataset

Surpasses Amazon Mechanical Turker performance

Demonstrates the effectiveness of word-level option comparison

Abstract

Multiple-choice reading comprehension (MCRC) is the task of selecting the correct answer from multiple options given a question and an article. Existing MCRC models typically either read each option independently or compute a fixed-length representation for each option before comparing them. However, humans typically compare the options at multiple-granularity level before reading the article in detail to make reasoning more efficient. Mimicking humans, we propose an option comparison network (OCN) for MCRC which compares options at word-level to better identify their correlations to help reasoning. Specially, each option is encoded into a vector sequence using a skimmer to retain fine-grained information as much as possible. An attention mechanism is leveraged to compare these sequences vector-by-vector to identify more subtle correlations between options, which is potentially valuable for reasoning. Experimental results on the human English exam MCRC dataset RACE show that our model outperforms existing methods significantly. Moreover, it is also the first model that surpasses Amazon Mechanical Turker performance on the whole dataset.