Measuring Perceived Trust in XAI-Assisted Decision-Making by Eliciting a Mental Model

TL;DR

This study introduces a new method using Fuzzy Cognitive Maps to quantify users' perceived trust in XAI models by analyzing their mental models and interpretation satisfaction, validated through medical expert evaluations.

Contribution

It presents a novel approach combining Fuzzy Cognitive Maps and interpretability ratings to measure perceived trust in XAI, incorporating mental model elicitation and fuzzy logic.

Findings

Quantified trust values correlate with medical experts' diagnostic performance.

The methodology effectively distinguishes trust levels based on mental model influences.

Fuzzy logic captures subjective influence strengths in mental models.

Abstract

This empirical study proposes a novel methodology to measure users' perceived trust in an Explainable Artificial Intelligence (XAI) model. To do so, users' mental models are elicited using Fuzzy Cognitive Maps (FCMs). First, we exploit an interpretable Machine Learning (ML) model to classify suspected COVID-19 patients into positive or negative cases. Then, Medical Experts' (MEs) conduct a diagnostic decision-making task based on their knowledge and then prediction and interpretations provided by the XAI model. In order to evaluate the impact of interpretations on perceived trust, explanation satisfaction attributes are rated by MEs through a survey. Then, they are considered as FCM's concepts to determine their influences on each other and, ultimately, on the perceived trust. Moreover, to consider MEs' mental subjectivity, fuzzy linguistic variables are used to determine the strength of influences. After reaching the steady state of FCMs, a quantified value is obtained to measure the perceived trust of each ME. The results show that the quantified values can determine whether MEs trust or distrust the XAI model. We analyze this behavior by comparing the quantified values with MEs' performance in completing diagnostic tasks.