Global Model Interpretation via Recursive Partitioning

TL;DR

This paper introduces a method to interpret black-box machine learning models globally by constructing a compact binary tree based on contribution matrices, revealing key decision rules and aiding understanding.

Contribution

It presents a novel approach to global model interpretation using recursive partitioning of contribution data to generate an interpretable decision tree.

Findings

Effective diagnosis of models across multiple tasks

Facilitates new knowledge discovery from model behavior

Simplifies understanding of complex black-box models

Abstract

In this work, we propose a simple but effective method to interpret black-box machine learning models globally. That is, we use a compact binary tree, the interpretation tree, to explicitly represent the most important decision rules that are implicitly contained in the black-box machine learning models. This tree is learned from the contribution matrix which consists of the contributions of input variables to predicted scores for each single prediction. To generate the interpretation tree, a unified process recursively partitions the input variable space by maximizing the difference in the average contribution of the split variable between the divided spaces. We demonstrate the effectiveness of our method in diagnosing machine learning models on multiple tasks. Also, it is useful for new knowledge discovery as such insights are not easily identifiable when only looking at single predictions. In general, our work makes it easier and more efficient for human beings to understand machine learning models.