Evaluating Trust in AI, Human, and Co-produced Feedback Among Undergraduate Students

TL;DR

This study compares undergraduate students' trust in AI, human, and co-produced feedback, revealing biases and the influence of AI experience on perceptions, highlighting the importance of source credibility and literacy for effective AI integration in education.

Contribution

It provides empirical evidence on students' trust and biases towards different feedback sources in higher education, emphasizing the role of source disclosure and AI experience.

Findings

Students preferred AI and co-produced feedback when source was blinded.

Disclosing feedback source increased bias against AI feedback.

AI experience improved feedback identification and trust.

Abstract

As generative AI models, particularly large language models (LLMs), transform educational feedback practices in higher education (HE) contexts, understanding students' perceptions of different sources of feedback becomes crucial for their effective implementation and adoption. This study addresses a critical gap by comparing undergraduate students' trust in LLM, human, and human-AI co-produced feedback in their authentic HE context. More specifically, through a within-subject experimental design involving 91 participants, we investigated factors that predict students' ability to distinguish between feedback types, their perceptions of feedback quality, and potential biases related to the source of feedback. Findings revealed that when the source was blinded, students generally preferred AI and co-produced feedback over human feedback regarding perceived usefulness and objectivity. However, they presented a strong bias against AI when the source of feedback was disclosed. In addition, only AI feedback suffered a decline in perceived genuineness when feedback sources were revealed, while co-produced feedback maintained its positive perception. Educational AI experience improved students' ability to identify LLM-generated feedback and increased their trust in all types of feedback. More years of students' experience using AI for general purposes were associated with lower perceived usefulness and credibility of feedback. These insights offer substantial evidence of the importance of source credibility and the need to enhance both feedback literacy and AI literacy to mitigate bias in student perceptions for AI-generated feedback to be adopted and impact education.