MEME: Generating RNN Model Explanations via Model Extraction

TL;DR

MEME is a novel method that extracts interpretable models from RNNs, enabling better understanding of their decision processes through human-understandable concepts and interactions, demonstrated on real-world datasets.

Contribution

We introduce MEME, a model extraction technique that approximates RNNs with interpretable models based on concepts and their interactions, enhancing explainability.

Findings

Effective interpretation of RNNs in case studies

Models reveal key concept interactions influencing decisions

Approach applicable to multivariate continuous data

Abstract

Recurrent Neural Networks (RNNs) have achieved remarkable performance on a range of tasks. A key step to further empowering RNN-based approaches is improving their explainability and interpretability. In this work we present MEME: a model extraction approach capable of approximating RNNs with interpretable models represented by human-understandable concepts and their interactions. We demonstrate how MEME can be applied to two multivariate, continuous data case studies: Room Occupation Prediction, and In-Hospital Mortality Prediction. Using these case-studies, we show how our extracted models can be used to interpret RNNs both locally and globally, by approximating RNN decision-making via interpretable concept interactions.