Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators

TL;DR

This study examines how trust in AI affects students' ability to appropriately rely on AI assistance during programming tasks, moderated by AI literacy and need for cognition.

Contribution

It reveals a non-linear relationship between trust and reliance, moderated by individual differences, emphasizing the importance of fostering reflective evaluation skills.

Findings

Higher trust correlates with lower appropriate reliance on AI.

AI literacy and need for cognition moderate trust-reliance relationship.

Students often over-rely on AI, especially with higher trust levels.

Abstract

As generative AI systems are integrated into educational settings, students often encounter AI-generated output while working through learning tasks, either by requesting help or through integrated tools. Trust in AI can influence how students interpret and use that output, including whether they evaluate it critically or exhibit overreliance. We investigate how students' trust relates to their appropriate reliance on an AI assistant during programming problem-solving tasks, and whether this relationship differs by learner characteristics. With 432 undergraduate participants, students' completed Python output-prediction problems while receiving recommendations and explanations from an AI chatbot, including accurate and intentionally misleading suggestions. We operationalize reliance behaviorally as the extent to which students' responses reflected appropriate use of the AI assistant's suggestions, accepting them when they were correct and rejecting them when they were incorrect. Pre- and post-task surveys assessed trust in the assistant, AI literacy, need for cognition, programming self-efficacy, and programming literacy. Results showed a non-linear relationship in which higher trust was associated with lower appropriate reliance, suggesting weaker discrimination between correct and incorrect recommendations. This relationship was significantly moderated by students' AI literacy and need for cognition. These findings highlight the need for future work on instructional and system supports that encourage more reflective evaluation of AI assistance during problem-solving.