Streamlining models with explanations in the learning loop

TL;DR

This paper introduces methods to incorporate local explanation insights into the training process, creating streamlined models that better align with explanation relevance while maintaining accuracy.

Contribution

It proposes two novel strategies, Iterative Dataset Weighting and Targeted Replacement Values, to integrate explanations into feature engineering for improved model interpretability.

Findings

Streamlined models achieve comparable accuracy to original black-box classifiers.

Explanations become more compact and relevant in the new models.

The methods enhance the interpretability of machine learning models.

Abstract

Several explainable AI methods allow a Machine Learning user to get insights on the classification process of a black-box model in the form of local linear explanations. With such information, the user can judge which features are locally relevant for the classification outcome, and get an understanding of how the model reasons. Standard supervised learning processes are purely driven by the original features and target labels, without any feedback loop informed by the local relevance of the features identified by the post-hoc explanations. In this paper, we exploit this newly obtained information to design a feature engineering phase, where we combine explanations with feature values. To do so, we develop two different strategies, named Iterative Dataset Weighting and Targeted Replacement Values, which generate streamlined models that better mimic the explanation process presented to the user. We show how these streamlined models compare to the original black-box classifiers, in terms of accuracy and compactness of the newly produced explanations.