Towards Rigorous Explainability by Feature Attribution

TL;DR

This paper discusses the shift from non-symbolic, less rigorous explainability methods to more rigorous symbolic approaches in machine learning, emphasizing the importance of provable feature attribution especially in high-stakes scenarios.

Contribution

It overviews efforts to replace non-rigorous methods like Shapley values with symbolic, provably rigorous techniques for feature importance in explainable AI.

Findings

Non-symbolic methods can mislead in high-stakes ML applications.

Symbolic methods offer a more rigorous alternative for feature attribution.

The paper highlights ongoing research towards symbolic explainability.

Abstract

For around a decade, non-symbolic methods have been the option of choice when explaining complex machine learning (ML) models. Unfortunately, such methods lack rigor and can mislead human decision-makers. In high-stakes uses of ML, the lack of rigor is especially problematic. One prime example of provable lack of rigor is the adoption of Shapley values in explainable artificial intelligence (XAI), with the tool SHAP being a ubiquitous example. This paper overviews the ongoing efforts towards using rigorous symbolic methods of XAI as an alternative to non-rigorous non-symbolic approaches, concretely for assigning relative feature importance.