A real-time metric of online engagement monitoring

TL;DR

This paper introduces a real-time, chapter-based engagement metric derived from virtual learning data, enabling early, scalable monitoring of student activity and performance in online courses.

Contribution

It reconceptualizes an existing engagement metric into a chapter-based version suitable for real-time tracking aligned with course structure.

Findings

Strong alignment with course-wide metrics from week 3

Comparable or better predictive validity for final grades

Early identification of low-performing students by midpoint

Abstract

Measuring online behavioural student engagement often relies on simple count indicators or retrospective, predictive methods, which present challenges for real-time application. To address these limitations, we reconceptualise an existing course-wide engagement metric to create a chapter-based version that aligns with the weekly structure of online courses. Derived directly from virtual learning environment log data, the new metric allows for cumulative, real-time tracking of student activity without requiring outcome data or model training. We evaluate the approach across three undergraduate statistics modules over two academic years, comparing it to the course-wide formulation to assess how the reconceptualisation influences what is measured. Results indicate strong alignment from as early as week 3, along with comparable or improved predictive validity for final grades in structured, lecture-based contexts. By the course midpoint, the weekly metric identifies as many low-performing students as are identifiable by the end of the course. While performance varies across modules, the chapter-based formulation offers a scalable and interpretable method for early engagement monitoring and student support.