Extracting Causal Relations in Deep Knowledge Tracing

TL;DR

This paper reveals that Deep Knowledge Tracing's success in educational modeling is primarily due to its ability to implicitly learn causal relationships between knowledge components, challenging previous explanations centered on bidirectional relationships.

Contribution

The study demonstrates that DKT models causal structures in knowledge tracing, introduces a method to extract causal relation DAGs from DKT representations, and provides empirical evidence supporting the causal interpretation.

Findings

DKT's performance aligns with causal structures in data

Pruning exercise graphs into DAGs maintains predictive accuracy

DKT's learned representations can be used to extract causal relations

Abstract

A longstanding goal in computational educational research is to develop explainable knowledge tracing (KT) models. Deep Knowledge Tracing (DKT), which leverages a Recurrent Neural Network (RNN) to predict student knowledge and performance on exercises, has been proposed as a major advancement over traditional KT methods. Several studies suggest that its performance gains stem from its ability to model bidirectional relationships between different knowledge components (KCs) within a course, enabling the inference of a student's understanding of one KC from their performance on others. In this paper, we challenge this prevailing explanation and demonstrate that DKT's strength lies in its implicit ability to model prerequisite relationships as a causal structure, rather than bidirectional relationships. By pruning exercise relation graphs into Directed Acyclic Graphs (DAGs) and training DKT on causal subsets of the Assistments dataset, we show that DKT's predictive capabilities align strongly with these causal structures. Furthermore, we propose an alternative method for extracting exercise relation DAGs using DKT's learned representations and provide empirical evidence supporting our claim. Our findings suggest that DKT's effectiveness is largely driven by its capacity to approximate causal dependencies between KCs rather than simple relational mappings.