Human Learning about AI

TL;DR

This paper investigates how humans project human-like qualities onto AI, affecting their evaluation and adoption, and demonstrates how anthropomorphic design can lead to misaligned beliefs and behaviors.

Contribution

It formalizes human projection of AI performance, tests its effects experimentally, and shows how anthropomorphic cues influence trust and adoption.

Findings

Humans overestimate AI on easy tasks and underestimate on hard ones.

Single ability index leads to all-or-nothing adoption based on partial AI performance.

Less human-like mistakes in AI reduce trust and engagement.

Abstract

We study \emph{Human Projection} (HP): people's tendency to evaluate AI using the same frameworks they use for humans -- treating features such as task difficulty and the reasonableness of mistakes as diagnostic of overall ability. We formalize HP and its consequences for equilibrium adoption, testing its predictions experimentally. First, people project human difficulty onto AI, overestimating performance on human-easy tasks, underestimating it on human-hard ones, and over-updating after easy failures and hard successes -- leading to systematic misspecification when AI performance is jagged rather than human-ordered. Second, HP interprets observed performance through a single ability index, inducing all-or-nothing adoption even when AI outperforms humans on only some tasks; experimentally stripping AI of human-like cues weakens cross-task generalization and reduces over-adoption. Finally, a field experiment with a parenting-advice chatbot shows that less humanly reasonable mistakes cause larger drops in trust and future engagement. Anthropomorphic AI design can amplify HP, misaligning beliefs and distorting adoption.