Enhancing Knowledge Tracing through Leakage-Free and Recency-Aware Embeddings

TL;DR

This paper introduces leakage-free, recency-aware embeddings for Knowledge Tracing that improve prediction accuracy by preventing label leakage and modeling learning dynamics more effectively.

Contribution

It proposes a masking technique to prevent label leakage and a recency encoding to capture learning dynamics, enhancing existing KT models.

Findings

Recency encoding outperforms traditional positional encodings.

Incorporating the proposed embeddings improves accuracy of KT models.

The approach is efficient and applicable across multiple benchmarks.

Abstract

Knowledge Tracing (KT) aims to predict a student's future performance based on their sequence of interactions with learning content. Many KT models rely on knowledge concepts (KCs), which represent the skills required for each item. However, some of these models are vulnerable to label leakage, in which input data inadvertently reveal the correct answer, particularly in datasets with multiple KCs per question. We propose a straightforward yet effective solution to prevent label leakage by masking ground-truth labels during input embedding construction in cases susceptible to leakage. To accomplish this, we introduce a dedicated MASK label, inspired by masked language modeling (e.g., BERT), to replace ground-truth labels. In addition, we introduce Recency Encoding, which encodes the step-wise distance between the current item and its most recent previous occurrence. This distance is important for modeling learning dynamics such as forgetting, which is a fundamental aspect of human learning, yet it is often overlooked in existing models. Recency Encoding demonstrates improved performance over traditional positional encodings on multiple KT benchmarks. We show that incorporating our embeddings into KT models like DKT, DKT+, AKT, and SAKT consistently improves prediction accuracy across multiple benchmarks. The approach is both efficient and widely applicable.