Understanding Roles and Entities: Datasets and Models for Natural Language Inference

TL;DR

This paper introduces two new datasets and a modified attention mechanism to improve neural models' understanding of entities and roles in natural language inference, addressing limitations of existing models.

Contribution

It provides novel datasets targeting entity and role understanding and proposes a modified attention mechanism that enhances model performance on these aspects.

Findings

Existing models perform poorly on new role/entity datasets

Modified attention improves understanding of entities and roles

Models maintain performance on traditional benchmarks

Abstract

We present two new datasets and a novel attention mechanism for Natural Language Inference (NLI). Existing neural NLI models, even though when trained on existing large datasets, do not capture the notion of entity and role well and often end up making mistakes such as "Peter signed a deal" can be inferred from "John signed a deal". The two datasets have been developed to mitigate such issues and make the systems better at understanding the notion of "entities" and "roles". After training the existing architectures on the new dataset we observe that the existing architectures does not perform well on one of the new benchmark. We then propose a modification to the "word-to-word" attention function which has been uniformly reused across several popular NLI architectures. The resulting architectures perform as well as their unmodified counterparts on the existing benchmarks and perform significantly well on the new benchmark for "roles" and "entities".