# Do Students Rely on AI? Analysis of Student-ChatGPT Conversations from a Field Study

## TL;DR

This study investigates how college students interact with ChatGPT-4 during quizzes, revealing low reliance, persistent negative patterns, and behavioral predictors of AI adoption, with implications for ethical and effective AI integration in education.

## Contribution

Introduces a novel four-stage reliance taxonomy and provides empirical insights into student-AI interaction patterns and reliance behaviors during early ChatGPT adoption.

## Key findings

- Students showed overall low reliance on AI.
- Negative reliance patterns persisted across interactions.
- Behavioral metrics predicted AI reliance effectively.

## Abstract

This study explores how college students interact with generative AI (ChatGPT-4) during educational quizzes, focusing on reliance and predictors of AI adoption. Conducted at the early stages of ChatGPT implementation, when students had limited familiarity with the tool, this field study analyzed 315 student-AI conversations during a brief, quiz-based scenario across various STEM courses. A novel four-stage reliance taxonomy was introduced to capture students' reliance patterns, distinguishing AI competence, relevance, adoption, and students' final answer correctness. Three findings emerged. First, students exhibited overall low reliance on AI and many of them could not effectively use AI for learning. Second, negative reliance patterns often persisted across interactions, highlighting students' difficulty in effectively shifting strategies after unsuccessful initial experiences. Third, certain behavioral metrics strongly predicted AI reliance, highlighting potential behavioral mechanisms to explain AI adoption. The study's findings underline critical implications for ethical AI integration in education and the broader field. It emphasizes the need for enhanced onboarding processes to improve student's familiarity and effective use of AI tools. Furthermore, AI interfaces should be designed with reliance-calibration mechanisms to enhance appropriate reliance. Ultimately, this research advances understanding of AI reliance dynamics, providing foundational insights for ethically sound and cognitively enriching AI practices.