Measuring the Impact of Student Gaming Behaviors on Learner Modeling

TL;DR

This paper investigates how gaming behaviors, viewed as data poisoning, affect knowledge tracing models in online education, revealing vulnerabilities and proposing detection methods to improve model robustness.

Contribution

It introduces a systematic framework for simulating gaming behaviors as data poisoning attacks and explores unsupervised detection approaches to enhance model resilience.

Findings

KT performance drops with random guessing behaviors

Systematic evaluation of gaming behavior impact

Unsupervised detection methods show promise

Abstract

The expansion of large-scale online education platforms has made vast amounts of student interaction data available for knowledge tracing (KT). KT models estimate students' concept mastery from interaction data, but their performance is sensitive to input data quality. Gaming behaviors, such as excessive hint use, may misrepresent students' knowledge and undermine model reliability. However, systematic investigations of how different types of gaming behaviors affect KT remain scarce, and existing studies rely on costly manual analysis that does not capture behavioral diversity. In this study, we conceptualize gaming behaviors as a form of data poisoning, defined as the deliberate submission of incorrect or misleading interaction data to corrupt a model's learning process. We design Data Poisoning Attacks (DPAs) to simulate diverse gaming patterns and systematically evaluate their impact on KT model performance. Moreover, drawing on advances in DPA detection, we explore unsupervised approaches to enhance the generalizability of gaming behavior detection. We find that KT models' performance tends to decrease especially in response to random guess behaviors. Our findings provide insights into the vulnerabilities of KT models and highlight the potential of adversarial methods for improving the robustness of learning analytics systems.