Student-AI Interaction: A Case Study of CS1 students

TL;DR

This case study explores how CS1 students interact with ChatGPT for programming tasks, revealing tendencies to rely heavily on AI without verification and discussing implications for learning and self-efficacy.

Contribution

It provides empirical insights into student-AI collaboration in programming education, highlighting usage patterns and potential challenges in student learning behaviors.

Findings

About one-third of students submitted full task descriptions to ChatGPT without prior effort.

Few students verified ChatGPT's solutions, indicating reliance on AI outputs.

Interactions influenced students' perceptions of their programming self-efficacy.

Abstract

The new capabilities of generative artificial intelligence tools Generative AI, such as ChatGPT, allow users to interact with the system in intuitive ways, such as simple conversations, and receive (mostly) good-quality answers. These systems can support students' learning objectives by providing accessible explanations and examples even with vague queries. At the same time, they can encourage undesired help-seeking behaviors by providing solutions to the students' homework. Therefore, it is important to better understand how students approach such tools and the potential issues such approaches might present for the learners. In this paper, we present a case study for understanding student-AI collaboration to solve programming tasks in the CS1 introductory programming course. To this end, we recruited a gender-balanced majority non-white set of 15 CS1 students at a large public university in the US. We observed them solving programming tasks. We used a mixed-method approach to study their interactions as they tackled Python programming tasks, focusing on when and why they used ChatGPT for problem-solving. We analyze and classify the questions submitted by the 15 participants to ChatGPT. Additionally, we analyzed user interaction patterns, their reactions to ChatGPT's responses, and the potential impacts of Generative AI on their perception of self-efficacy. Our results suggest that in about a third of the cases, the student attempted to complete the task by submitting the full description of the tasks to ChatGPT without making any effort on their own. We also observed that few students verified their solutions. We discuss the results and their potential implications.