A Unifying Framework for Unsupervised Concept Extraction

TL;DR

This paper introduces a unified theoretical framework for unsupervised concept extraction, framing it as a generative model identification problem, and provides a meta-theorem to simplify establishing guarantees.

Contribution

It presents a general meta-theorem for identifiability in concept extraction, simplifying proofs and enabling principled development of new approaches.

Findings

Meta-theorem reduces identifiability to set intersection characterization.

Framework applies to widely-used concept extraction methods.

Simplifies proving guarantees for unsupervised concept extraction.

Abstract

Techniques for concept extraction, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction.