Foundations of Symbolic Languages for Model Interpretability

TL;DR

This paper introduces FOIL, a logic-based declarative language for specifying explainability queries in ML models, analyzing its computational complexity and demonstrating its practical implementation.

Contribution

It develops a formal logic framework for ML interpretability, studies its computational properties, and provides a prototype implementation for practical use.

Findings

FOIL can express many explainability queries for ML models.

Evaluation complexity depends on model structure and query fragment.

Prototype implementation shows practical applicability.

Abstract

Several queries and scores have recently been proposed to explain individual predictions over ML models. Given the need for flexible, reliable, and easy-to-apply interpretability methods for ML models, we foresee the need for developing declarative languages to naturally specify different explainability queries. We do this in a principled way by rooting such a language in a logic, called FOIL, that allows for expressing many simple but important explainability queries, and might serve as a core for more expressive interpretability languages. We study the computational complexity of FOIL queries over two classes of ML models often deemed to be easily interpretable: decision trees and OBDDs. Since the number of possible inputs for an ML model is exponential in its dimension, the tractability of the FOIL evaluation problem is delicate but can be achieved by either restricting the structure of the models or the fragment of FOIL being evaluated. We also present a prototype implementation of FOIL wrapped in a high-level declarative language and perform experiments showing that such a language can be used in practice.