Visualizing Attention in Transformer-Based Language Representation Models

TL;DR

This paper introduces an open-source visualization tool for multi-head self-attention in Transformer models, enabling detailed interpretation at multiple levels and demonstrating its use on BERT and GPT-2 for bias detection and pattern analysis.

Contribution

The paper presents a novel visualization tool that extends previous work by providing multi-level attention visualization for Transformer models.

Findings

Effective visualization of attention at multiple levels

Identification of model bias and recurring patterns

Linking neurons to model behavior

Abstract

We present an open-source tool for visualizing multi-head self-attention in Transformer-based language representation models. The tool extends earlier work by visualizing attention at three levels of granularity: the attention-head level, the model level, and the neuron level. We describe how each of these views can help to interpret the model, and we demonstrate the tool on the BERT model and the OpenAI GPT-2 model. We also present three use cases for analyzing GPT-2: detecting model bias, identifying recurring patterns, and linking neurons to model behavior.