Exploring Interactions and Regulations in Collaborative Learning: An Interdisciplinary Multimodal Dataset

TL;DR

This paper introduces a new multimodal dataset capturing cognitive and emotional signals during collaborative learning among high school students, aiming to analyze how regulations influence interactions and learning outcomes.

Contribution

It presents a novel interdisciplinary multimodal dataset with designed interventions to study regulation effects in collaborative learning, including analysis of emotions, gestures, and multimodal signals.

Findings

Multimodal signals effectively reflect moments of regulation.

The dataset captures complex interactions in real-world educational settings.

Baseline models show the dataset's challenging nature for affective computing.

Abstract

Collaborative learning is an educational approach that enhances learning through shared goals and working together. Interaction and regulation are two essential factors related to the success of collaborative learning. Since the information from various modalities can reflect the quality of collaboration, a new multimodal dataset with cognitive and emotional triggers is introduced in this paper to explore how regulations affect interactions during the collaborative process. Specifically, a learning task with intentional interventions is designed and assigned to high school students aged 15 years old (N=81) in average. Multimodal signals, including video, Kinect, audio, and physiological data, are collected and exploited to study regulations in collaborative learning in terms of individual-participant-single-modality, individual-participant-multiple-modality, and multiple-participant-multiple-modality. Analysis of annotated emotions, body gestures, and their interactions indicates that our multimodal dataset with designed treatments could effectively examine moments of regulation in collaborative learning. In addition, preliminary experiments based on baseline models suggest that the dataset provides a challenging in-the-wild scenario, which could further contribute to the fields of education and affective computing.