The Impact of AI Generated Content on Decision Making for Topics Requiring Expertise
Shangqian Li, Tianwa Chen, Gianluca Demartini

TL;DR
This study investigates how AI-generated content influences online decision-making, especially on expert-level topics, revealing that users rely heavily on provided information and that AI content is as helpful as human sources, raising both potential and concerns.
Contribution
It provides empirical evidence on the influence of AI-generated content on decision-making for domain-specific topics requiring expertise, highlighting user reliance and perceptions.
Findings
Participants less likely to change opinions on expert topics.
AI-generated and human-written information are equally helpful.
Users rely on provided conclusions due to difficulty in independent reasoning.
Abstract
Modelling users' online decision-making and opinion change is a complex issue that needs to consider users' personal determinants, the nature of the topic and the information retrieval activities. Furthermore, generative-AIbased products like ChatGPT gradually become an essential element for the retrieval of online information. However, the interaction between domainspecific knowledge and AI-generated content during online decision-making is unclear. We conducted a lab-based explanatory sequential study with university students to overcome this research gap. In the experiment, we surveyed participants about a set of general domain topics that are easy to grasp and another set of domain-specific topics that require adequate levels of chemical science knowledge to fully comprehend. We provided participants with decision-supporting information that was either produced using generative AI…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Healthcare and Education · Educational Strategies and Epistemologies · Explainable Artificial Intelligence (XAI)
