Do You Trust ChatGPT? -- Perceived Credibility of Human and AI-Generated Content

TL;DR

This study investigates how people perceive the credibility of human versus AI-generated content across different interfaces, finding similar credibility perceptions but higher clarity and engagement ratings for AI content.

Contribution

It provides new insights into perceived credibility and engagement of AI-generated content, highlighting the need for critical evaluation of AI information sources.

Findings

Participants perceive similar credibility for human and AI content.

AI content is rated as clearer and more engaging.

Perceptions of trustworthiness and competence do not differ significantly.

Abstract

This paper examines how individuals perceive the credibility of content originating from human authors versus content generated by large language models, like the GPT language model family that powers ChatGPT, in different user interface versions. Surprisingly, our results demonstrate that regardless of the user interface presentation, participants tend to attribute similar levels of credibility. While participants also do not report any different perceptions of competence and trustworthiness between human and AI-generated content, they rate AI-generated content as being clearer and more engaging. The findings from this study serve as a call for a more discerning approach to evaluating information sources, encouraging users to exercise caution and critical thinking when engaging with content generated by AI systems.