InTraVisTo: Inside Transformer Visualisation Tool
Nicol\`o Brunello, Davide Rigamonti, Andrea Sassella, Vincenzo Scotti, Mark James Carman

TL;DR
InTraVisTo is a visualization tool that helps researchers understand the internal workings and information flow of Transformer-based large language models, aiding in interpretability and debugging.
Contribution
The paper introduces InTraVisTo, a novel visualization tool that displays internal states and information flow in Transformer models for better interpretability.
Findings
Provides detailed visualization of token embeddings at each layer
Displays information flow between model components using Sankey diagrams
Helps identify internal patterns and reasoning processes in LLMs
Abstract
The reasoning capabilities of Large Language Models (LLMs) have increased greatly over the last few years, as have their size and complexity. Nonetheless, the use of LLMs in production remains challenging due to their unpredictable nature and discrepancies that can exist between their desired behavior and their actual model output. In this paper, we introduce a new tool, InTraVisTo (Inside Transformer Visualisation Tool), designed to enable researchers to investigate and trace the computational process that generates each token in a Transformer-based LLM. InTraVisTo provides a visualization of both the internal state of the Transformer model (by decoding token embeddings at each layer of the model) and the information flow between the various components across the different layers of the model (using a Sankey diagram). With InTraVisTo, we aim to help researchers and practitioners better…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputational and Text Analysis Methods · Topic Modeling · Data Visualization and Analytics
