Navigating the Concept Space of Language Models

TL;DR

This paper introduces Concept Explorer, a scalable interactive system that organizes and navigates the concept space of language models using hierarchical embeddings, facilitating discovery and analysis of interpretable features.

Contribution

It presents a novel hierarchical embedding approach and an interactive tool for exploring language model features at multiple resolutions.

Findings

Reveals high-level structure in language model features

Identifies meaningful subclusters and rare concepts

Enhances interpretability of language model activations

Abstract

Sparse autoencoders (SAEs) trained on large language model activations output thousands of features that enable mapping to human-interpretable concepts. The current practice for analyzing these features primarily relies on inspecting top-activating examples, manually browsing individual features, or performing semantic search on interested concepts, which makes exploratory discovery of concepts difficult at scale. In this paper, we present Concept Explorer, a scalable interactive system for post-hoc exploration of SAE features that organizes concept explanations using hierarchical neighborhood embeddings. Our approach constructs a multi-resolution manifold over SAE feature embeddings and enables progressive navigation from coarse concept clusters to fine-grained neighborhoods, supporting discovery, comparison, and relationship analysis among concepts. We demonstrate the utility of Concept Explorer on SAE features extracted from SmolLM2, where it reveals coherent high-level structure, meaningful subclusters, and distinctive rare concepts that are hard to identify with existing workflows.