How Much Can We See? A Note on Quantifying Explainability of Machine Learning Models

TL;DR

This paper introduces a framework to quantify the explainability of machine learning models using partial dependence plots, helping determine how well these visualizations can explain model predictions.

Contribution

It develops a novel framework to measure the extent to which PDPs can explain black box models' predictions, addressing a key question in model interpretability.

Findings

Framework quantifies explainability of models

Enables judgment on sufficiency of explanations

Assists in evaluating visualization effectiveness

Abstract

One of the most popular approaches to understanding feature effects of modern black box machine learning models are partial dependence plots (PDP). These plots are easy to understand but only able to visualize low order dependencies. The paper is about the question 'How much can we see?': A framework is developed to quantify the explainability of arbitrary machine learning models, i.e. up to what degree the visualization as given by a PDP is able to explain the predictions of the model. The result allows for a judgement whether an attempt to explain a black box model is sufficient or not.