Discovering Concepts in Learned Representations using Statistical Inference and Interactive Visualization

TL;DR

This paper introduces two new methods, statistical inference and interactive visualization, to help users discover meaningful concepts in learned representations, addressing the challenge of navigating high-dimensional spaces.

Contribution

It proposes two novel approaches for concept discovery—one based on multiple hypothesis testing and another on interactive visualization—enhancing interpretability without full automation.

Findings

Methods show promise in identifying relevant concepts in high-dimensional spaces.

Simulation experiments validate the effectiveness of the proposed techniques.

Demo interface demonstrates practical application to real data.

Abstract

Concept discovery is one of the open problems in the interpretability literature that is important for bridging the gap between non-deep learning experts and model end-users. Among current formulations, concepts defines them by as a direction in a learned representation space. This definition makes it possible to evaluate whether a particular concept significantly influences classification decisions for classes of interest. However, finding relevant concepts is tedious, as representation spaces are high-dimensional and hard to navigate. Current approaches include hand-crafting concept datasets and then converting them to latent space directions; alternatively, the process can be automated by clustering the latent space. In this study, we offer another two approaches to guide user discovery of meaningful concepts, one based on multiple hypothesis testing, and another on interactive visualization. We explore the potential value and limitations of these approaches through simulation experiments and an demo visual interface to real data. Overall, we find that these techniques offer a promising strategy for discovering relevant concepts in settings where users do not have predefined descriptions of them, but without completely automating the process.