Calibrated Trust in Dealing with LLM Hallucinations: A Qualitative Study

TL;DR

This study explores how hallucinations in LLMs affect user trust, revealing that trust is context-sensitive and influenced by factors like perceived risk, prior experience, and intuition, leading to trust calibration rather than mistrust.

Contribution

It extends existing trust models by incorporating intuition and contextual factors, providing new insights into trust dynamics with hallucinating LLMs.

Findings

Hallucinations lead to context-sensitive trust calibration.

Trust factors include expectancy, prior experience, expertise, and intuition.

Contextual factors like perceived risk influence trust dynamics.

Abstract

Hallucinations are outputs by Large Language Models (LLMs) that are factually incorrect yet appear plausible [1]. This paper investigates how such hallucinations influence users' trust in LLMs and users' interaction with LLMs. To explore this in everyday use, we conducted a qualitative study with 192 participants. Our findings show that hallucinations do not result in blanket mistrust but instead lead to context-sensitive trust calibration. Building on the calibrated trust model by Lee & See [2] and Afroogh et al.'s trust-related factors [3], we confirm expectancy [3], [4], prior experience [3], [4], [5], and user expertise & domain knowledge [3], [4] as userrelated (human) trust factors, and identify intuition as an additional factor relevant for hallucination detection. Additionally, we found that trust dynamics are further influenced by contextual factors, particularly perceived risk [3] and decision stakes [6]. Consequently, we validate the recursive trust calibration process proposed by Bl\"obaum [7] and extend it by including intuition as a user-related trust factor. Based on these insights, we propose practical recommendations for responsible and reflective LLM use.