Modeling Collaborative Problem Solving Dynamics from Group Discourse: A Text-Mining Approach with Synergy Degree Model

TL;DR

This paper presents a computational framework combining automated discourse analysis with the Synergy Degree Model to quantify collaborative problem solving synergy from group communication data, enabling scalable and nuanced analysis.

Contribution

It introduces an AI-in-the-loop approach integrating classification models and the SDM to measure CPS synergy at multiple interaction levels, advancing scalable collaboration analytics.

Findings

GPT models showed superior precision for human-AI collaborative coding.

Synergy degree effectively distinguished collaborative quality among groups.

Task type influenced collaboration dynamics, with survey groups showing higher creation-order.

Abstract

Measuring collaborative problem solving (CPS) synergy remains challenging in learning analytics, as classical manual coding cannot capture emergent system-level dynamics. This study introduces a computational framework that integrates automated discourse analysis with the Synergy Degree Model (SDM) to quantify CPS synergy from group communication. Data were collected from 52 learners in 12 groups during a 5-week connectivist MOOC (cMOOC) activity. Nine classification models were applied to automatically identify ten CPS behaviors across four interaction levels: operation, wayfinding, sense-making, and creation. While BERT achieved the highest accuracy, GPT models demonstrated superior precision suitable for human-AI collaborative coding. Within the SDM framework, each interaction level was treated as a subsystem to compute group-level order parameters and derive synergy degrees. Permutation tests showed automated measures preserve construct validity, despite systematic biases at the subsystem level. Statistical analyses revealed significant task-type differences: survey study groups exhibited higher creation-order than mode study groups, suggesting "controlled disorder" may benefit complex problem solving. Importantly, synergy degree distinguished collaborative quality, ranging from excellent to failing groups. Findings establish synergy degree as a sensitive indicator of collaboration and demonstrate the feasibility of scaling fine-grained CPS analytics through AI-in-the-loop approaches.