Improved Explanatory Efficacy on Human Affect and Workload through Interactive Process in Artificial Intelligence

TL;DR

This paper introduces a new metric called explanatory efficacy to evaluate human-AI interaction quality, validated through a user study measuring affect, workload, and EEG signals during iterative AI tasks.

Contribution

The paper proposes a novel metric for assessing the cyclic relationship in human-AI interfaces and demonstrates its neural correlates through EEG analysis.

Findings

Significant hemispheric neural differences linked to explanatory efficacy

EEG signals can predict explanatory efficacy with 62.4% accuracy

Iterative AI tasks with higher explanatory efficacy improve user experience

Abstract

Despite recent advances in the field of explainable artificial intelligence systems, a concrete quantitative measure for evaluating the usability of such systems is nonexistent. Ensuring the success of an explanatory interface in interacting with users requires a cyclic, symbiotic relationship between human and artificial intelligence. We, therefore, propose explanatory efficacy, a novel metric for evaluating the strength of the cyclic relationship the interface exhibits. Furthermore, in a user study, we evaluated the perceived affect and workload and recorded the EEG signals of our participants as they interacted with our custom-built, iterative explanatory interface to build personalized recommendation systems. We found that systems for perceptually driven iterative tasks with greater explanatory efficacy are characterized by statistically significant hemispheric differences in neural signals with 62.4% accuracy, indicating the feasibility of neural correlates as a measure of explanatory efficacy. These findings are beneficial for researchers who aim to study the circular ecosystem of the human-artificial intelligence partnership.