Digging Deeper: Learning Multi-Level Concept Hierarchies

TL;DR

This paper introduces Multi-Level Concept Splitting (MLCS) and Deep-HiCEMs, enabling the discovery and utilization of multi-level concept hierarchies from coarse supervision, enhancing interpretability and intervention capabilities without sacrificing accuracy.

Contribution

It presents a novel method for learning multi-level concept hierarchies from limited supervision and a corresponding architecture for effective intervention at multiple abstraction levels.

Findings

MLCS discovers human-interpretable concepts absent during training.

Deep-HiCEMs maintain high accuracy while supporting multi-level interventions.

Hierarchical concepts improve interpretability and task performance.

Abstract

Although concept-based models promise interpretability by explaining predictions with human-understandable concepts, they typically rely on exhaustive annotations and treat concepts as flat and independent. To circumvent this, recent work has introduced Hierarchical Concept Embedding Models (HiCEMs) to explicitly model concept relationships, and Concept Splitting to discover sub-concepts using only coarse annotations. However, both HiCEMs and Concept Splitting are restricted to shallow hierarchies. We overcome this limitation with Multi-Level Concept Splitting (MLCS), which discovers multi-level concept hierarchies from only top-level supervision, and Deep-HiCEMs, an architecture that represents these discovered hierarchies and enables interventions at multiple levels of abstraction. Experiments across multiple datasets show that MLCS discovers human-interpretable concepts absent during training and that Deep-HiCEMs maintain high accuracy while supporting test-time concept interventions that can improve task performance.