Utilisation of Learning Analytics to Identify Students at Risk of Poor Academic Performance in Medical Schools

TL;DR

This study shows that online engagement in medical school, like test and discussion submissions, can predict student performance and help identify at-risk students early.

Contribution

The study demonstrates that learning analytics can inexpensively identify at-risk medical students through early online engagement metrics.

Findings

Higher test and discussion board submissions correlated with better exam scores (r = 0.281 to 0.324, p<0.001).

Engagement scores showed a small but significant link to academic performance (r = 0.202, p<0.001).

Early engagement metrics can predict underperformance before assessments.

Abstract

Introduction

Identifying students at risk of failure before they experience difficulties may considerably improve their outcomes. However, identification techniques can be costly, time-intensive, and of unknown efficacy. Medical educators need accessible and cost-effective ways of identifying at-risk students. The aim of this study was to investigate the relationship between student engagement in an online classroom and academic performance given the transition of many courses from in-person to online learning.

Methods

A retrospective study was conducted on a group of 235 students from the University of Edinburgh Bachelor of Medicine and Surgery (MBChB) in Year One for eight weeks from the start of term, September 2020. Purposive sampling was used. Data were collected on total test submissions, total discussion board submissions, engagement scores, and overall exam scores. Learning analytics on discussion board engagement were collected for new medical students before they had sat any summative assessment. Tests completed, discussion board posts made, and their total engagement score were correlated with their first summative assessment scores at the end of semester one.

Results

We found a statistically significant correlation between total test submissions, total discussion board submissions, engagement scores, and overall exam scores, with small-medium effects (r = 0.281, p<0.001) (r = 0.241, p<0.001), and (r = 0.202, p<0.001). Students with more test submissions, total discussion board submissions, and total engagement had a higher overall exam score. There was a statistically significant moderate correlation between total submissions and overall exam scores (r = 0.324, p<0.001).

Conclusions

Students who had a higher number of submissions were more likely to perform better on assessments. Early engagement correlates with performance. Learning analytics can help identify student underperformance before they undertake any assessment, and this can be done very inexpensively and with minimal staff resources if properly planned.

Figures