Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling

TL;DR

This paper introduces ReSA, a hybrid hard and soft attention mechanism that efficiently models long-range dependencies in sequence data, leading to state-of-the-art results in natural language inference tasks.

Contribution

The paper proposes a novel integrated attention model combining hard and soft attention, with a new training method for hard attention called reinforced sequence sampling (RSS).

Findings

ReSAN achieves state-of-the-art performance on SNLI and SICK datasets.

ReSA efficiently models sparse dependencies in long sequences.

The combined hard-soft attention improves training efficiency and effectiveness.

Abstract

Many natural language processing tasks solely rely on sparse dependencies between a few tokens in a sentence. Soft attention mechanisms show promising performance in modeling local/global dependencies by soft probabilities between every two tokens, but they are not effective and efficient when applied to long sentences. By contrast, hard attention mechanisms directly select a subset of tokens but are difficult and inefficient to train due to their combinatorial nature. In this paper, we integrate both soft and hard attention into one context fusion model, "reinforced self-attention (ReSA)", for the mutual benefit of each other. In ReSA, a hard attention trims a sequence for a soft self-attention to process, while the soft attention feeds reward signals back to facilitate the training of the hard one. For this purpose, we develop a novel hard attention called "reinforced sequence sampling (RSS)", selecting tokens in parallel and trained via policy gradient. Using two RSS modules, ReSA efficiently extracts the sparse dependencies between each pair of selected tokens. We finally propose an RNN/CNN-free sentence-encoding model, "reinforced self-attention network (ReSAN)", solely based on ReSA. It achieves state-of-the-art performance on both Stanford Natural Language Inference (SNLI) and Sentences Involving Compositional Knowledge (SICK) datasets.