Implicit N-grams Induced by Recurrence

TL;DR

This paper uncovers explainable n-gram-like components within RNN hidden states, demonstrating their role in modeling linguistic phenomena and contributing to RNN performance in NLP tasks, thus enhancing interpretability.

Contribution

It reveals that RNNs contain interpretable n-gram components, offering insights into their internal mechanisms and inspiring new architecture designs for sequential data.

Findings

Extracted n-gram features from RNNs relate to negation and intensification.

N-gram components alone can effectively perform sentiment analysis and language modeling.

These components significantly contribute to RNNs' overall performance.

Abstract

Although self-attention based models such as Transformers have achieved remarkable successes on natural language processing (NLP) tasks, recent studies reveal that they have limitations on modeling sequential transformations (Hahn, 2020), which may prompt re-examinations of recurrent neural networks (RNNs) that demonstrated impressive results on handling sequential data. Despite many prior attempts to interpret RNNs, their internal mechanisms have not been fully understood, and the question on how exactly they capture sequential features remains largely unclear. In this work, we present a study that shows there actually exist some explainable components that reside within the hidden states, which are reminiscent of the classical n-grams features. We evaluated such extracted explainable features from trained RNNs on downstream sentiment analysis tasks and found they could be used to model interesting linguistic phenomena such as negation and intensification. Furthermore, we examined the efficacy of using such n-gram components alone as encoders on tasks such as sentiment analysis and language modeling, revealing they could be playing important roles in contributing to the overall performance of RNNs. We hope our findings could add interpretability to RNN architectures, and also provide inspirations for proposing new architectures for sequential data.