Pattern-based Knowledge Component Extraction from Student Code Using Representation Learning

TL;DR

This paper introduces a pattern-based, data-driven method for discovering knowledge components in student programming solutions, improving student learning modeling and prediction accuracy.

Contribution

It proposes a novel framework combining code representation, latent space abstraction, and clustering to identify meaningful KCs from student code, enhancing knowledge tracing models.

Findings

Discovered KCs align with student struggles and mastery patterns.

Enhanced predictive performance in student modeling with pattern-based KCs.

Learning curve analysis confirms KCs' relevance to learning theory.

Abstract

Personalized instruction aims to provide learners with support that adapts to their individual knowledge and progress toward learning objectives. Discovering and tracing Knowledge Components (KCs) is an important step in building accurate models of student learning. However, KC discovery in computer science education is challenging due to the open-ended nature of programming, wide variability in student solutions, and intertwined use of programming structures in code. We address these challenges with a pattern-based KC discovery method that uses a data-driven approach to define KCs as recurring structural patterns in student code that reveal persistent patterns of struggle and mastery in students' solutions. We then evaluate the discovered KCs using expert evaluation and statistical student modeling to demonstrate their effectiveness in capturing student learning and struggles. We propose a framework for modeling students' learning by deriving pattern-based KCs from student code through a three-stage process. First, an attention-based code representation model identifies Abstract Syntax Tree subtrees most relevant to code correctness. Second, a Variational Autoencoder abstracts these subtrees into a smooth latent space, capturing structural similarity across student submissions. Third, the resulting representations are clustered into pattern-based KCs. To assess the effectiveness of pattern-based KCs for modeling students' learning, we adapt the Deep Knowledge Tracing model to incorporate these KCs, demonstrating significant improvements in predictive performance over baseline KT methods. Additionally, the learning curve analysis showed alignment between the derived KCs and learning theory.