Neural Models for Reasoning over Multiple Mentions using Coreference

TL;DR

This paper introduces a novel recurrent neural layer that leverages coreference annotations to better aggregate information from multiple mentions of the same entity, improving reading comprehension performance especially with limited training data.

Contribution

The paper proposes a coreference-biased recurrent layer that enhances neural reasoning over multiple entity mentions in text.

Findings

Improved performance on Wikihop, LAMBADA, and bAbi datasets

Significant gains when training data is scarce

Effective integration of coreference information into RNNs

Abstract

Many problems in NLP require aggregating information from multiple mentions of the same entity which may be far apart in the text. Existing Recurrent Neural Network (RNN) layers are biased towards short-term dependencies and hence not suited to such tasks. We present a recurrent layer which is instead biased towards coreferent dependencies. The layer uses coreference annotations extracted from an external system to connect entity mentions belonging to the same cluster. Incorporating this layer into a state-of-the-art reading comprehension model improves performance on three datasets -- Wikihop, LAMBADA and the bAbi AI tasks -- with large gains when training data is scarce.