Revealing AI Reasoning Increases Trust but Crowds Out Unique Human Knowledge

TL;DR

Revealing AI reasoning boosts trust but can overshadow unique human insights, potentially leading to over-reliance on AI in collaborative decision-making.

Contribution

This study empirically demonstrates that displaying AI reasoning increases trust but may reduce the use of valuable human knowledge, highlighting challenges in designing transparent AI systems.

Findings

AI reasoning display increases trust and agreement

Transparency induces over-trust, crowding out human knowledge

Implications for designing better human-AI collaboration systems

Abstract

Effective human-AI collaboration requires humans to accurately gauge AI capabilities and calibrate their trust accordingly. Humans often have context-dependent private information, referred to as Unique Human Knowledge (UHK), that is crucial for deciding whether to accept or override AI's recommendations. We examine how displaying AI reasoning affects trust and UHK utilization through a pre-registered, incentive-compatible experiment (N = 752). We find that revealing AI reasoning, whether brief or extensive, acts as a powerful persuasive heuristic that significantly increases trust and agreement with AI recommendations. Rather than helping participants appropriately calibrate their trust, this transparency induces over-trust that crowds out UHK utilization. Our results highlight the need for careful consideration when revealing AI reasoning and call for better information design in human-AI collaboration systems.