Knowledge Tracing in Programming Education Integrating Students' Questions

TL;DR

This paper introduces SQKT, a novel knowledge tracing model that incorporates students' questions and automatically extracted skill information to improve performance prediction in programming education, demonstrating significant accuracy improvements.

Contribution

The paper presents SQKT, a new model that leverages students' questions and semantic embeddings to enhance knowledge tracing in programming courses, addressing data scarcity and generalization.

Findings

33.1% absolute improvement in AUC over baselines

Effective in cross-domain and data-scarce settings

Enhances adaptive learning system design

Abstract

Knowledge tracing (KT) in programming education presents unique challenges due to the complexity of coding tasks and the diverse methods students use to solve problems. Although students' questions often contain valuable signals about their understanding and misconceptions, traditional KT models often neglect to incorporate these questions as inputs to address these challenges. This paper introduces SQKT (Students' Question-based Knowledge Tracing), a knowledge tracing model that leverages students' questions and automatically extracted skill information to enhance the accuracy of predicting students' performance on subsequent problems in programming education. Our method creates semantically rich embeddings that capture not only the surface-level content of the questions but also the student's mastery level and conceptual understanding. Experimental results demonstrate SQKT's superior performance in predicting student completion across various Python programming courses of differing difficulty levels. In in-domain experiments, SQKT achieved a 33.1\% absolute improvement in AUC compared to baseline models. The model also exhibited robust generalization capabilities in cross-domain settings, effectively addressing data scarcity issues in advanced programming courses. SQKT can be used to tailor educational content to individual learning needs and design adaptive learning systems in computer science education.