Hybrid LLM-Embedded Dialogue Agents for Learner Reflection: Designing Responsive and Theory-Driven Interactions

TL;DR

This paper introduces a hybrid dialogue system combining rule-based pedagogical structure with LLM responsiveness to enhance learner reflection in a robotics camp, balancing theoretical grounding with flexible, context-aware responses.

Contribution

It presents a novel hybrid framework that embeds LLMs within a theory-driven dialogue structure to improve educational interactions.

Findings

LLM-embedded dialogues foster richer reflections on goals and activities.

The hybrid system increased responsiveness but faced challenges with repetitiveness.

Misaligned prompts sometimes reduced learner engagement.

Abstract

Dialogue systems have long supported learner reflections, with theoretically grounded, rule-based designs offering structured scaffolding but often struggling to respond to shifts in engagement. Large Language Models (LLMs), in contrast, can generate context-sensitive responses but are not informed by decades of research on how learning interactions should be structured, raising questions about their alignment with pedagogical theories. This paper presents a hybrid dialogue system that embeds LLM responsiveness within a theory-aligned, rule-based framework to support learner reflections in a culturally responsive robotics summer camp. The rule-based structure grounds dialogue in self-regulated learning theory, while the LLM decides when and how to prompt deeper reflections, responding to evolving conversation context. We analyze themes across dialogues to explore how our hybrid system shaped learner reflections. Our findings indicate that LLM-embedded dialogues supported richer learner reflections on goals and activities, but also introduced challenges due to repetitiveness and misalignment in prompts, reducing engagement.