Modeling Engagement Signals in Technology-Enhanced Collaborative Learning: Toward AI-Ready Feedback

TL;DR

This paper introduces a lightweight, interpretable framework for modeling engagement in collaborative learning environments, supporting AI-driven feedback and emphasizing that surface behaviors may not accurately reflect true engagement.

Contribution

It proposes a novel composite signal index combining shared understanding and consensus signals, along with an AI-ready prototype for transparent engagement assessment.

Findings

Positive correlation between CSI and sustained motivation

Constructive feedback (Q3) is a promising regulatory cue

Silence does not imply disengagement, and frequent talk does not ensure depth

Abstract

Modeling engagement in collaborative learning remains challenging, especially in technology-enhanced environments where surface indicators such as participation frequency can be misleading. This study proposes a lightweight and interpretable framework that operationalizes shared understanding (Q2), consensus building (Q4), and sustained motivation (Q6) as observable behavioral signals. Q2 and Q4 were consolidated into a Composite Signal Index (CSI), which supports a quadrant diagnostic model with implications for teacher- and AI-driven feedback. Constructive feedback (Q3), while not included in the CSI calculation, emerged as a meaningful regulatory cue and a strong candidate feature for future NLP-based modeling. An exploratory validation was conducted in an adult ESL classroom using a structured three-phase collaborative task (rotating reading -> retelling -> consensus). Results showed a positive association between CSI and sustained motivation, while qualitative reflections highlighted the potential role of Q3 in supporting shared regulation. We also designed an AI-ready prototype that maps structured behavioral cues onto transparent decision rules for instructional support. The framework provides a scalable and equitable approach to engagement modeling, emphasizing that silence does not equal disengagement and that frequent talk does not guarantee cognitive depth.