Student Engagement Detection Using Emotion Analysis, Eye Tracking and Head Movement with Machine Learning

TL;DR

This paper presents a real-time student engagement detection system using webcam-based emotion, eye, and head movement analysis, effectively classifying engagement levels during e-learning sessions.

Contribution

It introduces a novel multimodal approach combining facial emotion, eye, and head movement data for real-time engagement detection using standard webcams.

Findings

Accurately classifies engagement levels in real-time

Higher concentration indexes correlate with better student scores

System works effectively in typical e-learning scenarios

Abstract

With the increase of distance learning, in general, and e-learning, in particular, having a system capable of determining the engagement of students is of primordial importance, and one of the biggest challenges, both for teachers, researchers and policy makers. Here, we present a system to detect the engagement level of the students. It uses only information provided by the typical built-in web-camera present in a laptop computer, and was designed to work in real time. We combine information about the movements of the eyes and head, and facial emotions to produce a concentration index with three classes of engagement: "very engaged", "nominally engaged" and "not engaged at all". The system was tested in a typical e-learning scenario, and the results show that it correctly identifies each period of time where students were "very engaged", "nominally engaged" and "not engaged at all". Additionally, the results also show that the students with best scores also have higher concentration indexes.