Understanding Recurrent Neural State Using Memory Signatures

TL;DR

This paper introduces a visualization technique for analyzing the internal memory states of LSTMs and GRUs, enabling better understanding of how these models encode historical information in language and acoustic tasks.

Contribution

The method trains decoders to predict past inputs, creating memory signatures that reveal how recurrent kernels encode history, advancing interpretability of end-to-end sequence models.

Findings

Reveals divergence in memory bases of recurrent kernels

Visualizes differences between n-gram and recurrent models

Extracts historical knowledge from end-to-end ASR layers

Abstract

We demonstrate a network visualization technique to analyze the recurrent state inside the LSTMs/GRUs used commonly in language and acoustic models. Interpreting intermediate state and network activations inside end-to-end models remains an open challenge. Our method allows users to understand exactly how much and what history is encoded inside recurrent state in grapheme sequence models. Our procedure trains multiple decoders that predict prior input history. Compiling results from these decoders, a user can obtain a signature of the recurrent kernel that characterizes its memory behavior. We demonstrate this method's usefulness in revealing information divergence in the bases of recurrent factorized kernels, visualizing the character-level differences between the memory of n-gram and recurrent language models, and extracting knowledge of history encoded in the layers of grapheme-based end-to-end ASR networks.