"Close...but not as good as an educator." -- Using ChatGPT to provide formative feedback in large-class collaborative learning

TL;DR

This study explores using ChatGPT to deliver personalized formative feedback in large-group collaborative learning, highlighting its potential and challenges in supporting health professionals' evaluation plan development.

Contribution

It demonstrates the feasibility of employing ChatGPT for formative feedback in real-time collaborative learning and offers insights for improving its integration and effectiveness.

Findings

Respondents found ChatGPT feedback generally favorable.

Only three groups actively used feedback to improve their work.

Learners' responses varied based on group dynamics and familiarity with ChatGPT.

Abstract

Delivering personalised, formative feedback to multiple problem-based learning groups in a short time period can be almost impossible. We employed ChatGPT to provide personalised formative feedback in a one-hour Zoom break-out room activity that taught practicing health professionals how to formulate evaluation plans for digital health initiatives. Learners completed an evaluation survey that included Likert scales and open-ended questions that were analysed. Half of the 44 survey respondents had never used ChatGPT before. Overall, respondents found the feedback favourable, described a wide range of group dynamics, and had adaptive responses to the feedback, yet only three groups used the feedback loop to improve their evaluation plans. Future educators can learn from our experience including engineering prompts, providing instructions on how to use ChatGPT, and scaffolding optimal group interactions with ChatGPT. Future researchers should explore the influence of ChatGPT on group dynamics and derive design principles for the use of ChatGPT in collaborative learning.