I Bet You Did Not Mean That: Testing Semantic Importance via Betting

TL;DR

This paper introduces a statistically rigorous framework for testing the importance of semantic concepts in black-box models, ensuring transparent and reliable interpretation through independence testing and significance control.

Contribution

It formalizes the statistical importance of semantic concepts using conditional independence and applies sequential kernelized independence testing to rank and validate concept importance.

Findings

Effective importance ranking on synthetic datasets

Successful application to diverse vision-language models

Framework provides statistical guarantees for interpretability

Abstract

Recent works have extended notions of feature importance to semantic concepts that are inherently interpretable to the users interacting with a black-box predictive model. Yet, precise statistical guarantees, such as false positive rate and false discovery rate control, are needed to communicate findings transparently and to avoid unintended consequences in real-world scenarios. In this paper, we formalize the global (i.e., over a population) and local (i.e., for a sample) statistical importance of semantic concepts for the predictions of opaque models by means of conditional independence, which allows for rigorous testing. We use recent ideas of sequential kernelized independence testing (SKIT) to induce a rank of importance across concepts, and showcase the effectiveness and flexibility of our framework on synthetic datasets as well as on image classification tasks using several and diverse vision-language models.