Automatic Rule Extraction from Long Short Term Memory Networks

TL;DR

This paper introduces a method to interpret LSTM models by extracting important word patterns and distilling them into rules, enabling simpler, rule-based approximations of complex NLP models.

Contribution

The paper presents a novel approach for extracting human-readable rules from LSTMs, enhancing interpretability in NLP tasks.

Findings

Extracted key word patterns that influence LSTM outputs

Constructed rule-based classifiers that approximate LSTM performance

Validated rules through quantitative comparison with original models

Abstract

Although deep learning models have proven effective at solving problems in natural language processing, the mechanism by which they come to their conclusions is often unclear. As a result, these models are generally treated as black boxes, yielding no insight of the underlying learned patterns. In this paper we consider Long Short Term Memory networks (LSTMs) and demonstrate a new approach for tracking the importance of a given input to the LSTM for a given output. By identifying consistently important patterns of words, we are able to distill state of the art LSTMs on sentiment analysis and question answering into a set of representative phrases. This representation is then quantitatively validated by using the extracted phrases to construct a simple, rule-based classifier which approximates the output of the LSTM.