Interpreting Blackbox Models via Model Extraction

TL;DR

This paper introduces a novel algorithm for extracting decision tree explanations from blackbox models, providing accurate, interpretable global explanations that help understand complex models in critical applications.

Contribution

The paper presents a new active sampling algorithm for decision tree extraction that improves approximation accuracy and interpretability of blackbox models.

Findings

Decision trees achieved higher accuracy than baselines in approximating blackbox models.

User study confirmed the interpretability of the extracted decision trees.

Insights from explanations included validated causal issues.

Abstract

Interpretability has become incredibly important as machine learning is increasingly used to inform consequential decisions. We propose to construct global explanations of complex, blackbox models in the form of a decision tree approximating the original model---as long as the decision tree is a good approximation, then it mirrors the computation performed by the blackbox model. We devise a novel algorithm for extracting decision tree explanations that actively samples new training points to avoid overfitting. We evaluate our algorithm on a random forest to predict diabetes risk and a learned controller for cart-pole. Compared to several baselines, our decision trees are both substantially more accurate and equally or more interpretable based on a user study. Finally, we describe several insights provided by our interpretations, including a causal issue validated by a physician.