Stochastic Answer Networks for Machine Reading Comprehension

TL;DR

This paper introduces a stochastic answer network (SAN) that enhances machine reading comprehension by employing stochastic dropout during training, leading to improved robustness and competitive results on multiple datasets.

Contribution

The paper presents a novel stochastic dropout technique in answer modules for multi-step reasoning, improving robustness over previous reinforcement learning approaches.

Findings

Achieves state-of-the-art or competitive results on SQuAD, Adversarial SQuAD, and MS MARCO datasets.

Demonstrates that stochastic dropout improves model robustness.

Simplifies multi-step reasoning without reinforcement learning.

Abstract

We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension. Compared to previous work such as ReasoNet which used reinforcement learning to determine the number of steps, the unique feature is the use of a kind of stochastic prediction dropout on the answer module (final layer) of the neural network during the training. We show that this simple trick improves robustness and achieves results competitive to the state-of-the-art on the Stanford Question Answering Dataset (SQuAD), the Adversarial SQuAD, and the Microsoft MAchine Reading COmprehension Dataset (MS MARCO).