# Multilingual, Multi-scale and Multi-layer Visualization of Intermediate Representations

## TL;DR

This paper introduces a web-based visualization tool for intermediate layer representations in sequence models like RNNs, CNNs, and Transformers, enabling better interpretability across languages and layers.

## Contribution

It presents a novel visualization tool that makes intermediate representations in sequence models more accessible and interpretable for multilingual and multi-layer architectures.

## Key findings

- Analyzes gender bias in contextual embeddings
- Visualizes multilingual representations at sentence and token levels
- Tracks evolution of representations across layers in translation models

## Abstract

The main alternatives nowadays to deal with sequences are Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN) architectures and the Transformer. In this context, RNN's, CNN's and Transformer have most commonly been used as an encoder-decoder architecture with multiple layers in each module. Far beyond this, these architectures are the basis for the contextual word embeddings which are revolutionizing most natural language downstream applications. However, intermediate layer representations in sequence-based architectures can be difficult to interpret. To make each layer representation within these architectures more accessible and meaningful, we introduce a web-based tool that visualizes them both at the sentence and token level. We present three use cases. The first analyses gender issues in contextual word embeddings. The second and third are showing multilingual intermediate representations for sentences and tokens and the evolution of these intermediate representations along the multiple layers of the decoder and in the context of multilingual machine translation.