Can Users Detect Biases or Factual Errors in Generated Responses in Conversational Information-Seeking?

TL;DR

This paper examines users' ability to identify biases and factual errors in conversational information-seeking responses, revealing challenges in detecting inaccuracies and the importance of response diversity for user satisfaction.

Contribution

The study provides empirical insights into user detection of biases and errors in conversational responses and highlights response diversity as a key factor for satisfaction.

Findings

Users find it easier to detect incomplete responses than biased or incorrect ones.

User satisfaction correlates more with response diversity than factual correctness.

The research identifies critical issues for improving conversational information-seeking systems.

Abstract

Information-seeking dialogues span a wide range of questions, from simple factoid to complex queries that require exploring multiple facets and viewpoints. When performing exploratory searches in unfamiliar domains, users may lack background knowledge and struggle to verify the system-provided information, making them vulnerable to misinformation. We investigate the limitations of response generation in conversational information-seeking systems, highlighting potential inaccuracies, pitfalls, and biases in the responses. The study addresses the problem of query answerability and the challenge of response incompleteness. Our user studies explore how these issues impact user experience, focusing on users' ability to identify biased, incorrect, or incomplete responses. We design two crowdsourcing tasks to assess user experience with different system response variants, highlighting critical issues to be addressed in future conversational information-seeking research. Our analysis reveals that it is easier for users to detect response incompleteness than query answerability and user satisfaction is mostly associated with response diversity, not factual correctness.