Improving understanding and trust in AI: How users benefit from interval-based counterfactual explanations

TL;DR

This study demonstrates that interval-based counterfactual explanations significantly improve user understanding and trust in AI models compared to other explanation types, highlighting the importance of explanation design and individual differences.

Contribution

It provides empirical evidence that interval counterfactual explanations outperform other explanation types in fostering understanding and trust in AI systems.

Findings

Interval explanations increase model understanding more than other types.

Interval explanations enhance demonstrated trust in AI.

Individual differences influence explanation effectiveness.

Abstract

Experimental user studies evaluating the effectiveness of different subtypes of post-hoc explanations for black-box models are largely nonexistent. Therefore, the aim of this study was to investigate and evaluate how different types of counterfactual explanations, namely single point explanations and interval-based explanations, affect both model understanding and (demonstrated) trust. We conducted an online user study using a within-subjects experimental design, where the experimental arms were (i) no explanation (control), (ii) feature importance scores, (iii) point counterfactual explanations, and (iv) interval counterfactual explanations. Our results clearly show the superiority of interval explanations over other tested explanation types in increasing both model understanding and demonstrated trust in the AI. We could not support findings of some previous studies showing an effect of point counterfactual explanations compared to the control group. Our results further highlight the role individual differences in, for example, cognitive style or personality, in explanation effectiveness.