Explanations of Black-Box Model Predictions by Contextual Importance and Utility

TL;DR

This paper introduces Contextual Importance and Utility, a novel explanation method for black-box models that provides understandable, instance-specific explanations without needing to make models interpretable, applicable to both linear and non-linear models.

Contribution

The paper proposes CI and CU concepts for explaining predictions, offering a generalizable, model-agnostic approach that produces visual and natural language explanations for end-users.

Findings

Effective explanations for linear and non-linear models demonstrated.

Applicable to end-users with visual and natural language formats.

Validated through car selection and Iris classification examples.

Abstract

The significant advances in autonomous systems together with an immensely wider application domain have increased the need for trustable intelligent systems. Explainable artificial intelligence is gaining considerable attention among researchers and developers to address this requirement. Although there is an increasing number of works on interpretable and transparent machine learning algorithms, they are mostly intended for the technical users. Explanations for the end-user have been neglected in many usable and practical applications. In this work, we present the Contextual Importance (CI) and Contextual Utility (CU) concepts to extract explanations that are easily understandable by experts as well as novice users. This method explains the prediction results without transforming the model into an interpretable one. We present an example of providing explanations for linear and non-linear models to demonstrate the generalizability of the method. CI and CU are numerical values that can be represented to the user in visuals and natural language form to justify actions and explain reasoning for individual instances, situations, and contexts. We show the utility of explanations in car selection example and Iris flower classification by presenting complete (i.e. the causes of an individual prediction) and contrastive explanation (i.e. contrasting instance against the instance of interest). The experimental results show the feasibility and validity of the provided explanation methods.