Why Groups Matter: Necessity of Group Structures in Attributions

TL;DR

This paper emphasizes the importance of incorporating natural feature group structures in explainable machine learning, especially in financial domains, to ensure explanations align with domain knowledge and regulatory requirements.

Contribution

It introduces the significance of feature group structures in financial datasets and proposes group-based explanations to improve consistency with domain knowledge.

Findings

Group structures are crucial for financial data explanations.

Standard explainability methods may produce inconsistent results without considering groups.

Group-based Shapley values offer more consistent explanations.

Abstract

Explainable machine learning methods have been accompanied by substantial development. Despite their success, the existing approaches focus more on the general framework with no prior domain expertise. High-stakes financial sectors have extensive domain knowledge of the features. Hence, it is expected that explanations of models will be consistent with domain knowledge to ensure conceptual soundness. In this work, we study the group structures of features that are naturally formed in the financial dataset. Our study shows the importance of considering group structures that conform to the regulations. When group structures are present, direct applications of explainable machine learning methods, such as Shapley values and Integrated Gradients, may not provide consistent explanations; alternatively, group versions of the Shapley value can provide consistent explanations. We contain detailed examples to concentrate on the practical perspective of our framework.