Generating Situated Reflection Triggers about Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative Learning

TL;DR

This paper explores how Large Language Models can generate context-aware reflection prompts to enhance collaborative learning in programming and database design tasks, demonstrating their potential to improve student engagement.

Contribution

The study presents a novel application of ChatGPT to generate situated reflection triggers in a collaborative learning environment, with an evaluation showing positive impact on student learning.

Findings

LLMs can generate highly situated reflection triggers

Contextualized prompts improve student engagement

Positive impact observed in pilot study

Abstract

An advantage of Large Language Models (LLMs) is their contextualization capability - providing different responses based on student inputs like solution strategy or prior discussion, to potentially better engage students than standard feedback. We present a design and evaluation of a proof-of-concept LLM application to offer students dynamic and contextualized feedback. Specifically, we augment an Online Programming Exercise bot for a college-level Cloud Computing course with ChatGPT, which offers students contextualized reflection triggers during a collaborative query optimization task in database design. We demonstrate that LLMs can be used to generate highly situated reflection triggers that incorporate details of the collaborative discussion happening in context. We discuss in depth the exploration of the design space of the triggers and their correspondence with the learning objectives as well as the impact on student learning in a pilot study with 34 students.