May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability

TL;DR

This paper explores how free-form conversational explanations can enhance user understanding, trust, and collaboration with AI models, demonstrating significant improvements over static explanations.

Contribution

It introduces the concept of conversational explanations in XAI, showing their benefits in improving user comprehension and trust compared to traditional static methods.

Findings

Conversations significantly improve user comprehension.

Conversations increase acceptance and trust in AI explanations.

Participants better identified the most accurate model after conversations.

Abstract

Research in explainable AI (XAI) aims to provide insights into the decision-making process of opaque AI models. To date, most XAI methods offer one-off and static explanations, which cannot cater to the diverse backgrounds and understanding levels of users. With this paper, we investigate if free-form conversations can enhance users' comprehension of static explanations, improve acceptance and trust in the explanation methods, and facilitate human-AI collaboration. Participants are presented with static explanations, followed by a conversation with a human expert regarding the explanations. We measure the effect of the conversation on participants' ability to choose, from three machine learning models, the most accurate one based on explanations and their self-reported comprehension, acceptance, and trust. Empirical results show that conversations significantly improve comprehension, acceptance, trust, and collaboration. Our findings highlight the importance of customized model explanations in the format of free-form conversations and provide insights for the future design of conversational explanations.