Do we need to go Deep? Knowledge Tracing with Big Data

TL;DR

This study compares deep learning and traditional models for student knowledge prediction using the large EdNet dataset, finding traditional logistic regression with engineered features outperforms deep models.

Contribution

The paper provides an empirical comparison of deep and traditional models on a large-scale educational dataset, highlighting the effectiveness of feature engineering.

Findings

Logistic regression with engineered features outperforms deep models.

Deep models do not necessarily outperform traditional models on large educational datasets.

Feature importance analysis reveals key factors influencing model predictions.

Abstract

Interactive Educational Systems (IES) enabled researchers to trace student knowledge in different skills and provide recommendations for a better learning path. To estimate the student knowledge and further predict their future performance, the interest in utilizing the student interaction data captured by IES to develop learner performance models is increasing rapidly. Moreover, with the advances in computing systems, the amount of data captured by these IES systems is also increasing that enables deep learning models to compete with traditional logistic models and Markov processes. However, it is still not empirically evident if these deep models outperform traditional models on the current scale of datasets with millions of student interactions. In this work, we adopt EdNet, the largest student interaction dataset publicly available in the education domain, to understand how accurately both deep and traditional models predict future student performances. Our work observes that logistic regression models with carefully engineered features outperformed deep models from extensive experimentation. We follow this analysis with interpretation studies based on Locally Interpretable Model-agnostic Explanation (LIME) to understand the impact of various features on best performing model pre-dictions.