Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Models

TL;DR

This study investigates how anthropomorphic cues like speech and pronouns influence user trust and perception of accuracy in large language models, revealing that speech and first-person references increase anthropomorphism and perceived accuracy.

Contribution

It provides empirical evidence on how modality and pronoun use affect anthropomorphism and trust in LLMs, informing responsible AI design.

Findings

Speech + text increases anthropomorphism and perceived accuracy.

First-person pronoun 'I' enhances perceived accuracy and reduces risk ratings.

Context-specific effects of pronoun use on trust and perception.

Abstract

People now regularly interface with Large Language Models (LLMs) via speech and text (e.g., Bard) interfaces. However, little is known about the relationship between how users anthropomorphize an LLM system (i.e., ascribe human-like characteristics to a system) and how they trust the information the system provides. Participants (n=2,165; ranging in age from 18-90 from the United States) completed an online experiment, where they interacted with a pseudo-LLM that varied in modality (text only, speech + text) and grammatical person ("I" vs. "the system") in its responses. Results showed that the "speech + text" condition led to higher anthropomorphism of the system overall, as well as higher ratings of accuracy of the information the system provides. Additionally, the first-person pronoun ("I") led to higher information accuracy and reduced risk ratings, but only in one context. We discuss these findings for their implications for the design of responsible, human-generative AI experiences.