CodeTailor: LLM-Powered Personalized Parsons Puzzles for Engaging Support While Learning Programming

TL;DR

CodeTailor is a system that uses large language models to create personalized Parsons puzzles, helping students engage more actively and reflectively in learning programming, while addressing challenges of AI-generated code misuse.

Contribution

This paper introduces CodeTailor, a novel LLM-powered system that generates personalized Parsons puzzles to enhance engagement and learning in programming education.

Findings

Participants found CodeTailor more engaging than just receiving code solutions.

CodeTailor improved students' application of scaffolded elements in posttests.

Students reported increased reflection and confidence using CodeTailor.

Abstract

Learning to program can be challenging, and providing high-quality and timely support at scale is hard. Generative AI and its products, like ChatGPT, can create a solution for most intro-level programming problems. However, students might use these tools to just generate code for them, resulting in reduced engagement and limited learning. In this paper, we present CodeTailor, a system that leverages a large language model (LLM) to provide personalized help to students while still encouraging cognitive engagement. CodeTailor provides a personalized Parsons puzzle to support struggling students. In a Parsons puzzle, students place mixed-up code blocks in the correct order to solve a problem. A technical evaluation with previous incorrect student code snippets demonstrated that CodeTailor could deliver high-quality (correct, personalized, and concise) Parsons puzzles based on their incorrect code. We conducted a within-subjects study with 18 novice programmers. Participants perceived CodeTailor as more engaging than just receiving an LLM-generated solution (the baseline condition). In addition, participants applied more supported elements from the scaffolded practice to the posttest when using CodeTailor than baseline. Overall, most participants preferred using CodeTailor versus just receiving the LLM-generated code for learning. Qualitative observations and interviews also provided evidence for the benefits of CodeTailor, including thinking more about solution construction, fostering continuity in learning, promoting reflection, and boosting confidence. We suggest future design ideas to facilitate active learning opportunities with generative AI techniques.