OCCAM: Open-set Causal Concept explAnation and Ontology induction for black-box vision Models

TL;DR

OCCAM is a framework that explains black-box vision models by discovering, localizing, and intervening on visual concepts, and inducing a concept ontology to reveal model biases and causal relations.

Contribution

It introduces a novel open-set approach for causal concept explanation and ontology induction in vision models, enhancing interpretability beyond local attributions.

Findings

OCCAM improves explanation quality in open-set black-box settings.

It induces a structured concept ontology capturing causal relations.

Experiments demonstrate richer insights than traditional attribution methods.

Abstract

Interpreting the decisions of deep image classifiers remains challenging, particularly in black-box settings where model internals are inaccessible. We introduce OCCAM, a framework for open-set causal concept explanation and ontology induction in vision models. OCCAM discovers visual concepts in an open-set manner, localizes them via text-guided segmentation, and performs object-level interventions by removing concepts to measure changes in class confidence, estimating each concept's causal contribution. Beyond local explanations, OCCAM aggregates interventional evidence across a dataset to induce a structured concept ontology that captures how classifiers globally organize visual concepts. Reasoning over this ontology reveals consistent dependencies between concepts, exposes latent causal relations, and uncovers systematic model biases. Experiments on Broden and ImageNet-S across multiple classifiers show that OCCAM improves explanation quality in open-set black-box settings while providing richer global insight than per-image attribution methods.