An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comprehension Tasks

TL;DR

This paper empirically evaluates the effectiveness of multiple-turn reasoning strategies in reading comprehension tasks, demonstrating that flexible, multi-turn approaches outperform single-turn methods across datasets.

Contribution

It introduces an end-to-end neural RC model with reinforcement learning to dynamically control reasoning turns, showing improved performance over fixed strategies.

Findings

Multiple-turn reasoning outperforms single-turn reasoning.

Flexible turn control improves overall accuracy.

Achieves competitive results on SQuAD and MS MARCO.

Abstract

Reading comprehension (RC) is a challenging task that requires synthesis of information across sentences and multiple turns of reasoning. Using a state-of-the-art RC model, we empirically investigate the performance of single-turn and multiple-turn reasoning on the SQuAD and MS MARCO datasets. The RC model is an end-to-end neural network with iterative attention, and uses reinforcement learning to dynamically control the number of turns. We find that multiple-turn reasoning outperforms single-turn reasoning for all question and answer types; further, we observe that enabling a flexible number of turns generally improves upon a fixed multiple-turn strategy. %across all question types, and is particularly beneficial to questions with lengthy, descriptive answers. We achieve results competitive to the state-of-the-art on these two datasets.