# Visualizing RNN States with Predictive Semantic Encodings

## TL;DR

This paper introduces a visualization technique for RNN hidden states that provides semantic insights into their functioning, aiding interpretability in natural language processing tasks.

## Contribution

It presents a novel visual method for understanding RNN hidden states through semantic encodings, facilitating model interpretability.

## Key findings

- Enables comparison of hidden states across the model
- Provides high-level semantic intuition of RNNs
- Demonstrated on language modeling task

## Abstract

Recurrent Neural Networks are an effective and prevalent tool used to model sequential data such as natural language text. However, their deep nature and massive number of parameters pose a challenge for those intending to study precisely how they work. We present a visual technique that gives a high level intuition behind the semantics of the hidden states within Recurrent Neural Networks. This semantic encoding allows for hidden states to be compared throughout the model independent of their internal details. The proposed technique is displayed in a proof of concept visualization tool which is demonstrated to visualize the natural language processing task of language modelling.