SMACE: A New Method for the Interpretability of Composite Decision Systems

TL;DR

SMACE is a novel interpretability method designed for composite decision systems, combining geometric decision rule analysis with existing ML interpretability techniques to produce meaningful feature rankings tailored to end users.

Contribution

It introduces SMACE, a semi-model-agnostic approach that effectively explains composite decision systems, outperforming existing methods on tabular data.

Findings

SMACE provides more meaningful feature importance rankings.

Existing model-agnostic methods perform poorly on composite systems.

SMACE improves interpretability for complex decision-making processes.

Abstract

Interpretability is a pressing issue for decision systems. Many post hoc methods have been proposed to explain the predictions of a single machine learning model. However, business processes and decision systems are rarely centered around a unique model. These systems combine multiple models that produce key predictions, and then apply decision rules to generate the final decision. To explain such decisions, we propose the Semi-Model-Agnostic Contextual Explainer (SMACE), a new interpretability method that combines a geometric approach for decision rules with existing interpretability methods for machine learning models to generate an intuitive feature ranking tailored to the end user. We show that established model-agnostic approaches produce poor results on tabular data in this setting, in particular giving the same importance to several features, whereas SMACE can rank them in a meaningful way.