The Placebo Effect of Artificial Intelligence in Human-Computer Interaction

TL;DR

This study demonstrates that user expectations of AI support can create a placebo effect, improving perceived and actual performance in human-computer interaction, which has implications for evaluating AI interfaces.

Contribution

The paper provides empirical evidence of a placebo effect in AI-supported interfaces and discusses its impact on user performance and technology assessment.

Findings

Belief in AI support boosts performance expectations.

Expectations positively correlate with actual task success.

Placebo effects can bias user-centered evaluations of AI systems.

Abstract

In medicine, patients can obtain real benefits from a sham treatment. These benefits are known as the placebo effect. We report two experiments (Experiment I: N=369; Experiment II: N=100) demonstrating a placebo effect in adaptive interfaces. Participants were asked to solve word puzzles while being supported by no system or an adaptive AI interface. All participants experienced the same word puzzle difficulty and had no support from an AI throughout the experiments. Our results showed that the belief of receiving adaptive AI support increases expectations regarding the participant's own task performance, sustained after interaction. These expectations were positively correlated to performance, as indicated by the number of solved word puzzles. We integrate our findings into technological acceptance theories and discuss implications for the future assessment of AI-based user interfaces and novel technologies. We argue that system descriptions can elicit placebo effects through user expectations biasing the results of user-centered studies.