Reasoning about Entailment with Neural Attention

TL;DR

This paper introduces a neural attention-based model for textual entailment that outperforms previous neural and feature-engineered classifiers, demonstrating effective reasoning over word and phrase pairs.

Contribution

It presents the first end-to-end differentiable neural model with attention mechanisms that achieves state-of-the-art accuracy on entailment tasks.

Findings

Outperforms previous neural models and feature-based classifiers

Demonstrates effective reasoning over word and phrase entailments

Achieves state-of-the-art accuracy on a large entailment dataset

Abstract

While most approaches to automatically recognizing entailment relations have used classifiers employing hand engineered features derived from complex natural language processing pipelines, in practice their performance has been only slightly better than bag-of-word pair classifiers using only lexical similarity. The only attempt so far to build an end-to-end differentiable neural network for entailment failed to outperform such a simple similarity classifier. In this paper, we propose a neural model that reads two sentences to determine entailment using long short-term memory units. We extend this model with a word-by-word neural attention mechanism that encourages reasoning over entailments of pairs of words and phrases. Furthermore, we present a qualitative analysis of attention weights produced by this model, demonstrating such reasoning capabilities. On a large entailment dataset this model outperforms the previous best neural model and a classifier with engineered features by a substantial margin. It is the first generic end-to-end differentiable system that achieves state-of-the-art accuracy on a textual entailment dataset.