Explainability for Transparent Conversational Information-Seeking

TL;DR

This paper investigates how different explanation methods and presentation modes in conversational search systems affect user trust and response usefulness, emphasizing transparency about sources, confidence, and limitations.

Contribution

It introduces a user study analyzing the impact of explanation quality and presentation on user perception in conversational information-seeking.

Findings

Noisy explanations reduce user ratings.

Explanation quality influences perceived usefulness.

Presentation format has inconclusive effects.

Abstract

The increasing reliance on digital information necessitates advancements in conversational search systems, particularly in terms of information transparency. While prior research in conversational information-seeking has concentrated on improving retrieval techniques, the challenge remains in generating responses useful from a user perspective. This study explores different methods of explaining the responses, hypothesizing that transparency about the source of the information, system confidence, and limitations can enhance users' ability to objectively assess the response. By exploring transparency across explanation type, quality, and presentation mode, this research aims to bridge the gap between system-generated responses and responses verifiable by the user. We design a user study to answer questions concerning the impact of (1) the quality of explanations enhancing the response on its usefulness and (2) ways of presenting explanations to users. The analysis of the collected data reveals lower user ratings for noisy explanations, although these scores seem insensitive to the quality of the response. Inconclusive results on the explanations presentation format suggest that it may not be a critical factor in this setting.