Automated Visual Attention Detection using Mobile Eye Tracking in Behavioral Classroom Studies

TL;DR

This paper introduces an automated, minimally supervised system combining mobile eye tracking and face recognition to identify which students teachers focus on in classrooms, aiding educational research and teacher training.

Contribution

It presents a novel pipeline that integrates face detection, recognition, and gaze data to automatically determine teacher focus with minimal manual annotation.

Findings

Achieved approximately 0.7 accuracy in U-shaped classrooms.

Achieved approximately 0.9 accuracy in small classrooms.

Demonstrated the method's applicability across different classroom setups.

Abstract

Teachers' visual attention and its distribution across the students in classrooms can constitute important implications for student engagement, achievement, and professional teacher training. Despite that, inferring the information about where and which student teachers focus on is not trivial. Mobile eye tracking can provide vital help to solve this issue; however, the use of mobile eye tracking alone requires a significant amount of manual annotations. To address this limitation, we present an automated processing pipeline concept that requires minimal manually annotated data to recognize which student the teachers focus on. To this end, we utilize state-of-the-art face detection models and face recognition feature embeddings to train face recognition models with transfer learning in the classroom context and combine these models with the teachers' gaze from mobile eye trackers. We evaluated our approach with data collected from four different classrooms, and our results show that while it is possible to estimate the visually focused students with reasonable performance in all of our classroom setups, U-shaped and small classrooms led to the best results with accuracies of approximately 0.7 and 0.9, respectively. While we did not evaluate our method for teacher-student interactions and focused on the validity of the technical approach, as our methodology does not require a vast amount of manually annotated data and offers a non-intrusive way of handling teachers' visual attention, it could help improve instructional strategies, enhance classroom management, and provide feedback for professional teacher development.