Real-Time Multimodal Data Collection Using Smartwatches and Its Visualization in Education

TL;DR

This paper introduces two tools, Watch-DMLT and ViSeDOPS, for real-time multimodal data collection and visualization using smartwatches in educational settings, enabling scalable analysis of physiological and behavioral data.

Contribution

The paper presents novel tools for synchronized multimodal data acquisition and visualization in education, addressing scalability and real-time analysis challenges.

Findings

Successful deployment with 65 students and 16 smartwatches

Collected data includes heart rate, motion, gaze, and video

Demonstrated system's feasibility and utility in real classrooms

Abstract

Wearable sensors, such as smartwatches, have become increasingly prevalent across domains like healthcare, sports, and education, enabling continuous monitoring of physiological and behavioral data. In the context of education, these technologies offer new opportunities to study cognitive and affective processes such as engagement, attention, and performance. However, the lack of scalable, synchronized, and high-resolution tools for multimodal data acquisition continues to be a significant barrier to the widespread adoption of Multimodal Learning Analytics in real-world educational settings. This paper presents two complementary tools developed to address these challenges: Watch-DMLT, a data acquisition application for Fitbit Sense 2 smartwatches that enables real-time, multi-user monitoring of physiological and motion signals; and ViSeDOPS, a dashboard-based visualization system for analyzing synchronized multimodal data collected during oral presentations. We report on a classroom deployment involving 65 students and up to 16 smartwatches, where data streams including heart rate, motion, gaze, video, and contextual annotations were captured and analyzed. Results demonstrate the feasibility and utility of the proposed system for supporting fine-grained, scalable, and interpretable Multimodal Learning Analytics in real learning environments.