Automatically Assessing Students Performance with Smartphone Data

TL;DR

This study uses smartphone data and machine learning to predict student performance, demonstrating that models trained on weekly data and combined with historical information can achieve over 90% accuracy, even across different contexts.

Contribution

The paper introduces a new dataset collected via a smartphone app and proposes a pipeline that enhances prediction accuracy using historical student data and ensemble voting methods.

Findings

Weekly data windows improve prediction accuracy.

Models can generalize across different contexts, including pandemic conditions.

Ensemble methods and historical data increase accuracy beyond 90%.

Abstract

As the number of smart devices that surround us increases, so do the opportunities to create smart socially-aware systems. In this context, mobile devices can be used to collect data about students and to better understand how their day-to-day routines can influence their academic performance. Moreover, the Covid-19 pandemic led to new challenges and difficulties, also for students, with considerable impact on their lifestyle. In this paper we present a dataset collected using a smartphone application (ISABELA), which include passive data (e.g., activity and location) as well as self-reported data from questionnaires. We present several tests with different machine learning models, in order to classify students' performance. These tests were carried out using different time windows, showing that weekly time windows lead to better prediction and classification results than monthly time windows. Furthermore, it is shown that the created models can predict student performance even with data collected from different contexts, namely before and during the Covid-19 pandemic. SVMs, XGBoost and AdaBoost-SAMME with Random Forest were found to be the best algorithms, showing an accuracy greater than 78%. Additionally, we propose a pipeline that uses a decision level median voting algorithm to further improve the models' performance, by using historic data from the students to further improve the prediction. Using this pipeline, it is possible to further increase the performance of the models, with some of them obtaining an accuracy greater than 90%.