Trust Me on This: A User Study of Trustworthiness for RAG Responses

TL;DR

This paper examines how different explanation types affect user trust in RAG system responses, revealing that trust depends on explanation, clarity, and user knowledge, not just response quality.

Contribution

It provides empirical evidence on the influence of explanations on user trust in RAG responses, highlighting factors beyond objective quality.

Findings

Explanations guide users toward higher quality responses.

User trust is influenced by response clarity and actionability.

Prior knowledge affects trust judgments.

Abstract

The integration of generative AI into information access systems often presents users with synthesized answers that lack transparency. This study investigates how different types of explanations can influence user trust in responses from retrieval-augmented generation systems. We conducted a controlled, two-stage user study where participants chose the more trustworthy response from a pair-one objectively higher quality than the other-both with and without one of three explanation types: (1) source attribution, (2) factual grounding, and (3) information coverage. Our results show that while explanations significantly guide users toward selecting higher quality responses, trust is not dictated by objective quality alone: Users' judgments are also heavily influenced by response clarity, actionability, and their own prior knowledge.