Tracing Prompt-Level Trajectories to Understand Student Learning with AI in Programming Education

TL;DR

This study analyzes how students interact with AI tools in programming education, revealing diverse prompting strategies and their relation to learning outcomes, with implications for designing better AI-assisted learning systems.

Contribution

It introduces a detailed analysis of student-AI interaction trajectories in programming tasks, highlighting their role in understanding student learning and guiding system design.

Findings

Most students copied AI code directly, but many refined prompts iteratively.

Prompting trajectories reflect students' self-regulation and learning orientation.

Interaction patterns correlate with assignment success and course performance.

Abstract

As AI tools such as ChatGPT enter programming classrooms, students encounter differing rules across courses and instructors, which shape how they use AI and leave them with unequal capabilities for leveraging it. We investigate how students engaged with AI in an introductory Python assignment, analyzing student-LLM chat histories and final code submissions from 163 students. We examined prompt-level strategies, traced trajectories of interaction, and compared AI-generated code with student submissions. We identified trajectories ranging from full delegation to iterative refinement, with hybrid forms in between. Although most students directly copied AI-generated code in their submission, many students scaffolded the code generation through iterative refinement. We also contrasted interaction patterns with assignment outcomes and course performance. Our findings show that prompting trajectories serve as promising windows into students' self-regulation and learning orientation. We draw design implications for educational AI systems that promote personalized and productive student-AI collaborative learning.