Semantic match: Debugging feature attribution methods in XAI for healthcare

TL;DR

This paper addresses the reliability of feature attribution methods in Explainable AI for healthcare, emphasizing the importance of semantic match between explanations and human understanding, especially for data with meaningful low-level features like EHRs.

Contribution

It introduces a semantic match framework for evaluating feature attribution methods in healthcare data and proposes a testing procedure for their reliability.

Findings

Semantic match is achievable in healthcare data with meaningful low-level features.

Criticism of feature attribution methods often misapplies to healthcare data, especially tabular data.

A procedure to test semantic match in explanations is proposed.

Abstract

The recent spike in certified Artificial Intelligence (AI) tools for healthcare has renewed the debate around adoption of this technology. One thread of such debate concerns Explainable AI (XAI) and its promise to render AI devices more transparent and trustworthy. A few voices active in the medical AI space have expressed concerns on the reliability of Explainable AI techniques and especially feature attribution methods, questioning their use and inclusion in guidelines and standards. Despite valid concerns, we argue that existing criticism on the viability of post-hoc local explainability methods throws away the baby with the bathwater by generalizing a problem that is specific to image data. We begin by characterizing the problem as a lack of semantic match between explanations and human understanding. To understand when feature importance can be used reliably, we introduce a distinction between feature importance of low- and high-level features. We argue that for data types where low-level features come endowed with a clear semantics, such as tabular data like Electronic Health Records (EHRs), semantic match can be obtained, and thus feature attribution methods can still be employed in a meaningful and useful way. Finally, we sketch a procedure to test whether semantic match has been achieved.