From Prediction to Application: Language Model-based Code Knowledge Tracing with Domain Adaptive Pre-Training and Automatic Feedback System with Pedagogical Prompting for Comprehensive Programming Education

TL;DR

This paper introduces CodeLKT, a language model-based approach to programming knowledge tracing, enhanced by domain adaptive pre-training and an automatic feedback system, significantly improving interpretability and cross-domain transfer in programming education.

Contribution

It proposes CodeLKT, integrating language models with adaptive pre-training and feedback mechanisms, advancing programming knowledge tracing beyond traditional methods.

Findings

CodeLKT outperforms existing KT models in accuracy.

Domain adaptive pre-training enhances cross-domain transfer.

The integrated system provides personalized, in-depth feedback for learners.

Abstract

Knowledge Tracing (KT) is a critical component in online learning, but traditional approaches face limitations in interpretability and cross-domain adaptability. This paper introduces Language Model-based Code Knowledge Tracing (CodeLKT), an innovative application of Language model-based Knowledge Tracing (LKT) to programming education. CodeLKT leverages pre-trained language models to process learning data, demonstrating superior performance over existing KT and Code KT models. We explore Domain Adaptive Pre-Training (DAPT) and Task Adaptive Pre-Training (TAPT), showing enhanced performance in the coding domain and investigating cross-domain transfer between mathematics and coding. Additionally, we present an theoretically-informed integrated system combining CodeLKT with large language models to generate personalized, in-depth feedback to support students' programming learning. This work advances the field of Code Knowledge Tracing by expanding the knowledge base with language model-based approach and offering practical implications for programming education through data-informed feedback.