Student-centric Model of Learning Management System Activity and Academic Performance: from Correlation to Causation

TL;DR

This paper introduces a student-centric framework for analyzing LMS activity data that uncovers causal relationships, particularly highlighting login volume as a key factor influencing academic performance, enabling targeted interventions.

Contribution

It presents a novel causal analysis approach for LMS data that shifts focus from course-centered to student-centered insights, expanding behavior indicators including login timing.

Findings

Login volume is strongly correlated with academic performance.

Login volume has a causal effect on student performance, especially for low performers.

Expanded LMS behavior indicators, like login time-of-day, enhance understanding of student engagement.

Abstract

In recent years, there is a lot of interest in modeling students' digital traces in Learning Management System (LMS) to understand students' learning behavior patterns including aspects of meta-cognition and self-regulation, with the ultimate goal to turn those insights into actionable information to support students to improve their learning outcomes. In achieving this goal, however, there are two main issues that need to be addressed given the existing literature. Firstly, most of the current work is course-centered (i.e. models are built from data for a specific course) rather than student-centered; secondly, a vast majority of the models are correlational rather than causal. Those issues make it challenging to identify the most promising actionable factors for intervention at the student level where most of the campus-wide academic support is designed for. In this paper, we explored a student-centric analytical framework for LMS activity data that can provide not only correlational but causal insights mined from observational data. We demonstrated this approach using a dataset of 1651 computing major students at a public university in the US during one semester in the Fall of 2019. This dataset includes students' fine-grained LMS interaction logs and administrative data, e.g. demographics and academic performance. In addition, we expand the repository of LMS behavior indicators to include those that can characterize the time-of-the-day of login (e.g. chronotype). Our analysis showed that student login volume, compared with other login behavior indicators, is both strongly correlated and causally linked to student academic performance, especially among students with low academic performance. We envision that those insights will provide convincing evidence for college student support groups to launch student-centered and targeted interventions that are effective and scalable.