Foundations of Interpretable Models

TL;DR

This paper critiques current interpretability definitions, proposes a comprehensive and actionable new definition, and offers a blueprint and open-source library to guide the design of interpretable models.

Contribution

It introduces a new, general interpretability definition that informs model design and provides a practical blueprint and open-source tools for building interpretable models.

Findings

New interpretability definition is general and actionable.

Provides a blueprint for designing interpretable models.

Introduces an open-source library supporting interpretable data structures.

Abstract

We argue that existing definitions of interpretability are not actionable in that they fail to inform users about general, sound, and robust interpretable model design. This makes current interpretability research fundamentally ill-posed. To address this issue, we propose a definition of interpretability that is general, simple, and subsumes existing informal notions within the interpretable AI community. We show that our definition is actionable, as it directly reveals the foundational properties, underlying assumptions, principles, data structures, and architectural features necessary for designing interpretable models. Building on this, we propose a general blueprint for designing interpretable models and introduce the first open-sourced library with native support for interpretable data structures and processes.