ULTra: Unveiling Latent Token Interpretability in Transformer-Based Understanding and Segmentation

TL;DR

ULTra is a novel framework that interprets transformer embeddings to reveal semantic patterns, enabling unsupervised segmentation and model explanation without fine-tuning, achieving state-of-the-art results.

Contribution

ULTra introduces a new unsupervised interpretability framework for transformers that uncovers semantic structures and improves segmentation performance without model fine-tuning.

Findings

Achieves state-of-the-art unsupervised semantic segmentation performance.

Effectively interprets latent token representations in CV and NLP models.

Demonstrates broad applicability in model explanation and interpretability tasks.

Abstract

Transformers have revolutionized Computer Vision (CV) through self-attention mechanisms. However, their complexity makes latent token representations difficult to interpret. We introduce ULTra, a framework for interpreting Transformer embeddings and uncovering meaningful semantic patterns within them. ULTra enables unsupervised semantic segmentation using pre-trained models without requiring fine-tuning. Additionally, we propose a self-supervised training approach that refines segmentation performance by learning an external transformation matrix without modifying the underlying model. Our method achieves state-of-the-art performance in unsupervised semantic segmentation, outperforming existing segmentation methods. Furthermore, we validate ULTra for model interpretation on both synthetic and real-world scenarios, including Object Selection and interpretable text summarization using LLMs, demonstrating its broad applicability in explaining the semantic structure of latent token representations.