On the Opportunities of Large Language Models for Programming Process Data

TL;DR

This paper explores how large language models can be utilized to analyze programming process data, offering new opportunities for automated feedback and summarization in computing education.

Contribution

It presents a discussion on leveraging LLMs for analyzing programming processes and provides a case study demonstrating automatic summarization and feedback generation.

Findings

LLMs can effectively summarize programming processes.

LLMs enable automated, formative feedback in programming education.

The approach advances automated analysis of programming data.

Abstract

Computing educators and researchers have used programming process data to understand how programs are constructed and what sorts of problems students struggle with. Although such data shows promise for using it for feedback, fully automated programming process feedback systems have still been an under-explored area. The recent emergence of large language models (LLMs) have yielded additional opportunities for researchers in a wide variety of fields. LLMs are efficient at transforming content from one format to another, leveraging the body of knowledge they have been trained with in the process. In this article, we discuss opportunities of using LLMs for analyzing programming process data. To complement our discussion, we outline a case study where we have leveraged LLMs for automatically summarizing the programming process and for creating formative feedback on the programming process. Overall, our discussion and findings highlight that the computing education research and practice community is again one step closer to automating formative programming process-focused feedback.