A Machine Learning Approach to Assess Student Group Collaboration Using Individual Level Behavioral Cues

TL;DR

This paper introduces a deep learning method to automatically evaluate student group collaboration quality using individual behavioral cues, addressing data scarcity and class imbalance with innovative augmentation and loss functions.

Contribution

It presents a novel approach combining Mixup data augmentation and ordinal loss to assess collaboration quality from individual student behaviors in large classrooms.

Findings

Effective handling of limited data and class imbalance.

Improved accuracy with Mixup and ordinal loss functions.

Potential for scalable, automated classroom assessment.

Abstract

K-12 classrooms consistently integrate collaboration as part of their learning experiences. However, owing to large classroom sizes, teachers do not have the time to properly assess each student and give them feedback. In this paper we propose using simple deep-learning-based machine learning models to automatically determine the overall collaboration quality of a group based on annotations of individual roles and individual level behavior of all the students in the group. We come across the following challenges when building these models: 1) Limited training data, 2) Severe class label imbalance. We address these challenges by using a controlled variant of Mixup data augmentation, a method for generating additional data samples by linearly combining different pairs of data samples and their corresponding class labels. Additionally, the label space for our problem exhibits an ordered structure. We take advantage of this fact and also explore using an ordinal-cross-entropy loss function and study its effects with and without Mixup.