SINKT: A Structure-Aware Inductive Knowledge Tracing Model with Large Language Model

TL;DR

SINKT introduces a structure-aware, inductive knowledge tracing model leveraging large language models to better handle data sparsity, cold start issues, and complex concept-question relationships in intelligent tutoring systems.

Contribution

The paper presents the first inductive KT model that uses LLMs to explicitly model structural relationships and semantic information in a heterogeneous graph of concepts and questions.

Findings

SINKT outperforms 12 existing transductive KT models on four real-world datasets.

SINKT effectively addresses data sparsity and cold start problems.

The model demonstrates strong performance in inductive knowledge tracing tasks.

Abstract

Knowledge Tracing (KT) aims to determine whether students will respond correctly to the next question, which is a crucial task in intelligent tutoring systems (ITS). In educational KT scenarios, transductive ID-based methods often face severe data sparsity and cold start problems, where interactions between individual students and questions are sparse, and new questions and concepts consistently arrive in the database. In addition, existing KT models only implicitly consider the correlation between concepts and questions, lacking direct modeling of the more complex relationships in the heterogeneous graph of concepts and questions. In this paper, we propose a Structure-aware Inductive Knowledge Tracing model with large language model (dubbed SINKT), which, for the first time, introduces large language models (LLMs) and realizes inductive knowledge tracing. Firstly, SINKT utilizes LLMs to introduce structural relationships between concepts and constructs a heterogeneous graph for concepts and questions. Secondly, by encoding concepts and questions with LLMs, SINKT incorporates semantic information to aid prediction. Finally, SINKT predicts the student's response to the target question by interacting with the student's knowledge state and the question representation. Experiments on four real-world datasets demonstrate that SINKT achieves state-of-the-art performance among 12 existing transductive KT models. Additionally, we explore the performance of SINKT on the inductive KT task and provide insights into various modules.