Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference

TL;DR

This paper presents a recurrent neural network-based sentence encoder with gated attention that achieves top performance in natural language inference tasks, demonstrating strong generalization across domains and state-of-the-art results on SNLI without cross-sentence attention.

Contribution

The paper introduces a novel intra-sentence gated-attention mechanism within a recurrent neural network encoder for natural language inference, achieving top results in shared tasks and on SNLI.

Findings

Achieved 74.9% accuracy on RepEval 2017 in-domain and cross-domain tests.

Obtained 85.5% accuracy on SNLI without cross-sentence attention.

Model generalizes well across different datasets.

Abstract

The RepEval 2017 Shared Task aims to evaluate natural language understanding models for sentence representation, in which a sentence is represented as a fixed-length vector with neural networks and the quality of the representation is tested with a natural language inference task. This paper describes our system (alpha) that is ranked among the top in the Shared Task, on both the in-domain test set (obtaining a 74.9% accuracy) and on the cross-domain test set (also attaining a 74.9% accuracy), demonstrating that the model generalizes well to the cross-domain data. Our model is equipped with intra-sentence gated-attention composition which helps achieve a better performance. In addition to submitting our model to the Shared Task, we have also tested it on the Stanford Natural Language Inference (SNLI) dataset. We obtain an accuracy of 85.5%, which is the best reported result on SNLI when cross-sentence attention is not allowed, the same condition enforced in RepEval 2017.