# Visual Reasoning of Feature Attribution with Deep Recurrent Neural Networks

## TL;DR

This paper introduces a visual analytics method to interpret deep RNNs by revealing attention mechanisms and variable attributions across sequences, aiding data scientists in understanding model decisions and dynamics.

## Contribution

The paper presents a novel visual analytics approach for interpreting RNN attention and feature attribution over sequences, enhancing understanding of deep RNNs' decision processes.

## Key findings

- Helps data scientists understand RNN attention dynamics
- Reveals variable attributions at each sequence position
- Guides model development through interpretability

## Abstract

Deep Recurrent Neural Network (RNN) has gained popularity in many sequence classification tasks. Beyond predicting a correct class for each data instance, data scientists also want to understand what differentiating factors in the data have contributed to the classification during the learning process. We present a visual analytics approach to facilitate this task by revealing the RNN attention for all data instances, their temporal positions in the sequences, and the attribution of variables at each value level. We demonstrate with real-world datasets that our approach can help data scientists to understand such dynamics in deep RNNs from the training results, hence guiding their modeling process.