Early Performance Prediction using Interpretable Patterns in Programming Process Data

TL;DR

This paper presents an early performance prediction model for programming students using interpretable behavioral patterns extracted from detailed log data, achieving high accuracy and providing actionable insights for instructors.

Contribution

It introduces a novel differential sequence mining approach to extract interpretable, predictive behavior patterns from programming logs, improving early performance prediction.

Findings

Predicts final programming performance with 79% accuracy using early assignment data.

Extracted patterns are interpretable and linked to effective or ineffective programming behaviors.

Outperforms baseline models in early prediction accuracy.

Abstract

Instructors have limited time and resources to help struggling students, and these resources should be directed to the students who most need them. To address this, researchers have constructed models that can predict students' final course performance early in a semester. However, many predictive models are limited to static and generic student features (e.g. demographics, GPA), rather than computing-specific evidence that assesses a student's progress in class. Many programming environments now capture complete time-stamped records of students' actions during programming. In this work, we leverage this rich, fine-grained log data to build a model to predict student course outcomes. From the log data, we extract patterns of behaviors that are predictive of students' success using an approach called differential sequence mining. We evaluate our approach on a dataset from 106 students in a block-based, introductory programming course. The patterns extracted from our approach can predict final programming performance with 79% accuracy using only the first programming assignment, outperforming two baseline methods. In addition, we show that the patterns are interpretable and correspond to concrete, effective -- and ineffective -- novice programming behaviors. We also discuss these patterns and their implications for classroom instruction.