A Decomposable Attention Model for Natural Language Inference

TL;DR

This paper introduces a simple, attention-based neural model for natural language inference that achieves state-of-the-art results efficiently without relying on word order, and can be further improved with intra-sentence attention.

Contribution

The paper presents a decomposable attention model that simplifies NLI, reduces parameters, and improves performance without using word order information.

Findings

Achieves state-of-the-art results on SNLI dataset

Uses significantly fewer parameters than previous models

Further improvements with intra-sentence attention

Abstract

We propose a simple neural architecture for natural language inference. Our approach uses attention to decompose the problem into subproblems that can be solved separately, thus making it trivially parallelizable. On the Stanford Natural Language Inference (SNLI) dataset, we obtain state-of-the-art results with almost an order of magnitude fewer parameters than previous work and without relying on any word-order information. Adding intra-sentence attention that takes a minimum amount of order into account yields further improvements.