Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks

TL;DR

This paper presents a new method for interpreting LSTM cell dynamics at the cellular level, enabling identification of influential neurons and understanding their impact on network performance.

Contribution

The paper introduces a systematic pipeline for response characterization of individual LSTM cells, linking dynamical properties to network accuracy and capacity.

Findings

Identifies neurons with insightful dynamics

Quantifies relationships between dynamics and accuracy

Demonstrates scalability across benchmark datasets

Abstract

In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses. As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution. Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets.