Who Do LLMs Trust? Human Experts Matter More Than Other LLMs
Anooshka Bajaj, Zoran Tiganj

TL;DR
This study shows that large language models are more influenced by human expert feedback than by other LLMs, indicating a credibility-based social influence similar to humans across various decision tasks.
Contribution
It demonstrates that instruction-tuned LLMs prioritize human expert input over other LLMs, revealing a credibility-sensitive influence pattern in AI models.
Findings
Models conform more to human expert responses, even when incorrect.
Models revise answers more towards experts than other LLMs.
Expert framing strongly influences LLM decision-making.
Abstract
Large language models (LLMs) increasingly operate in environments where they encounter social information such as other agents' answers, tool outputs, or human recommendations. In humans, such inputs influence judgments in ways that depend on the source's credibility and the strength of consensus. This paper investigates whether LLMs exhibit analogous patterns of influence and whether they privilege feedback from humans over feedback from other LLMs. Across three binary decision-making tasks, reading comprehension, multi-step reasoning, and moral judgment, we present four instruction-tuned LLMs with prior responses attributed either to friends, to human experts, or to other LLMs. We manipulate whether the group is correct and vary the group size. In a second experiment, we introduce direct disagreement between a single human and a single LLM. Across tasks, models conform significantly…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsExplainable Artificial Intelligence (XAI) · Artificial Intelligence in Healthcare and Education · Topic Modeling
