Understanding with toy surrogate models in machine learning

TL;DR

This paper explores the role of toy surrogate models like rule lists and decision trees in helping non-experts understand complex machine learning models globally, highlighting their unique theoretical and practical significance.

Contribution

It introduces the concept of toy surrogate models in ML as a new object of study for understanding opaque models, bridging scientific toy models and ML interpretability.

Findings

Toy surrogate models simplify understanding of complex ML models.

They highlight relevant features and their effects on outputs.

The paper offers a theoretical account of understanding via these models.

Abstract

In the natural and social sciences, it is common to use toy models -- extremely simple and highly idealized representations -- to understand complex phenomena. Some of the simple surrogate models used to understand opaque machine learning (ML) models, such as rule lists and sparse decision trees, bear some resemblance to scientific toy models. They allow non-experts to understand how an opaque ML model works globally via a much simpler model that highlights the most relevant features of the input space and their effect on the output. The obvious difference is that the common target of a toy and a full-scale model in the sciences is some phenomenon in the world, while the target of a surrogate model is another model. This essential difference makes toy surrogate models (TSMs) a new object of study for theories of understanding, one that is not easily accommodated under current analyses. This paper provides an account of what it means to understand an opaque ML model globally with the aid of such simple models.