Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretation

TL;DR

This paper introduces a statistically rigorous framework for interpreting CLIP embeddings by identifying meaningful concepts with theoretical guarantees, improving interpretability and robustness in model analysis.

Contribution

It presents a hypothesis testing framework for detecting rotation-sensitive structures in CLIP embeddings and a post-hoc concept decomposition method with theoretical guarantees.

Findings

Achieves a 22.6% increase in worst-group accuracy after removing spurious concepts

Balances reconstruction accuracy with interpretability effectively

Outperforms existing methods in concept robustness and reproducibility

Abstract

Concept-based approaches, which aim to identify human-understandable concepts within a model's internal representations, are a promising method for interpreting embeddings from deep neural network models, such as CLIP. While these approaches help explain model behavior, current methods lack statistical rigor, making it challenging to validate identified concepts and compare different techniques. To address this challenge, we introduce a hypothesis testing framework that quantifies rotation-sensitive structures within the CLIP embedding space. Once such structures are identified, we propose a post-hoc concept decomposition method. Unlike existing approaches, it offers theoretical guarantees that discovered concepts represent robust, reproducible patterns (rather than method-specific artifacts) and outperforms other techniques in terms of reconstruction error. Empirically, we demonstrate that our concept-based decomposition algorithm effectively balances reconstruction accuracy with concept interpretability and helps mitigate spurious cues in data. Applied to a popular spurious correlation dataset, our method yields a 22.6% increase in worst-group accuracy after removing spurious background concepts.