Tracing Mathematical Proficiency Through Problem-Solving Processes

TL;DR

This paper introduces a novel Knowledge Tracing framework that incorporates students' problem-solving processes and uses large language models to improve prediction accuracy and interpretability in mathematical proficiency assessment.

Contribution

It proposes KT-PSP, a new dataset and StatusKT, a three-stage LLM pipeline that captures multidimensional student proficiency and enhances explainability in knowledge tracing.

Findings

StatusKT outperforms existing KT methods in prediction accuracy.

The framework provides interpretable explanations of student proficiency.

Experimental results validate the effectiveness of incorporating problem-solving processes.

Abstract

Knowledge Tracing (KT) aims to model student's knowledge state and predict future performance to enable personalized learning in Intelligent Tutoring Systems. However, traditional KT methods face fundamental limitations in explainability, as they rely solely on the response correctness, neglecting the rich information embedded in students' problem-solving processes. To address this gap, we propose Knowledge Tracing Leveraging Problem-Solving Process (KT-PSP), which incorporates students' problem-solving processes to capture the multidimensional aspects of mathematical proficiency. We also introduce KT-PSP-25, a new dataset specifically designed for the KT-PSP. Building on this, we present StatusKT, a KT framework that employs a teacher-student-teacher three-stage LLM pipeline to extract students' MP as intermediate signals. In this pipeline, the teacher LLM first extracts problem-specific proficiency indicators, then a student LLM generates responses based on the student's solution process, and a teacher LLM evaluates these responses to determine mastery of each indicator. The experimental results on KT-PSP-25 demonstrate that StatusKT improves the prediction performance of existing KT methods. Moreover, StatusKT provides interpretable explanations for its predictions by explicitly modeling students' mathematical proficiency.