LIMIS: Locally Interpretable Modeling using Instance-wise Subsampling

TL;DR

LIMIS is a novel framework that enhances locally interpretable models' fidelity by instance-wise subsampling guided by policy gradients, achieving near black-box accuracy and outperforming existing methods.

Contribution

The paper introduces LIMIS, a new approach that uses policy gradient-based instance selection to improve the fidelity of locally interpretable models.

Findings

LIMIS achieves high fidelity close to black-box models.

LIMIS significantly outperforms existing locally interpretable models.

LIMIS maintains prediction accuracy comparable to black-box models.

Abstract

Understanding black-box machine learning models is crucial for their widespread adoption. Learning globally interpretable models is one approach, but achieving high performance with them is challenging. An alternative approach is to explain individual predictions using locally interpretable models. For locally interpretable modeling, various methods have been proposed and indeed commonly used, but they suffer from low fidelity, i.e. their explanations do not approximate the predictions well. In this paper, our goal is to push the state-of-the-art in high-fidelity locally interpretable modeling. We propose a novel framework, Locally Interpretable Modeling using Instance-wise Subsampling (LIMIS). LIMIS utilizes a policy gradient to select a small number of instances and distills the black-box model into a low-capacity locally interpretable model using those selected instances. Training is guided with a reward obtained directly by measuring the fidelity of the locally interpretable models. We show on multiple tabular datasets that LIMIS near-matches the prediction accuracy of black-box models, significantly outperforming state-of-the-art locally interpretable models in terms of fidelity and prediction accuracy.