Deep Trustworthy Knowledge Tracing

TL;DR

This paper critiques the reliability issues of deep learning-based knowledge tracing models in education, identifies key limitations, and proposes a regularization method to enhance their trustworthiness and adaptability.

Contribution

It introduces a novel regularization technique to improve the reliability and interpretability of deep knowledge tracing models in real educational settings.

Findings

Addresses knowledge state update failure

Mitigates catastrophic forgetting in DLKT

Achieves more trustworthy and adaptable models

Abstract

Knowledge tracing (KT), a key component of an intelligent tutoring system, is a machine learning technique that estimates the mastery level of a student based on his/her past performance. The objective of KT is to predict a student's response to the next question. Compared with traditional KT models, deep learning-based KT (DLKT) models show better predictive performance because of the representation power of deep neural networks. Various methods have been proposed to improve the performance of DLKT, but few studies have been conducted on the reliability of DLKT. In this work, we claim that the existing DLKTs are not reliable in real education environments. To substantiate the claim, we show limitations of DLKT from various perspectives such as knowledge state update failure, catastrophic forgetting, and non-interpretability. We then propose a novel regularization to address these problems. The proposed method allows us to achieve trustworthy DLKT. In addition, the proposed model which is trained on scenarios with forgetting can also be easily extended to scenarios without forgetting.