Tailoring Scaffolding to Diagnostic Strategies: Theory-Informed LLM-Based Agents

TL;DR

This paper proposes a theory-informed, adaptive scaffolding approach using LLM-based agents aligned with diagnostic strategies, aiming to improve personalized learning in complex environments.

Contribution

It introduces a KLI-informed hybrid LLM agent that adapts scaffolding based on diagnostic strategies, enhancing pedagogical alignment and personalization.

Findings

Different diagnostic strategies benefit from distinct scaffolding forms.

The KLI-informed hybrid LLM agent aligns scaffolding with strategy demands.

Adaptive scaffolding potentially improves learning outcomes.

Abstract

Learning analytics systems increasingly integrate large language models (LLMs) to provide adaptive scaffolding in complex learning environments, yet personalization is often driven by global instructional choices rather than principled alignment with learning theory, limiting effectiveness and pedagogical grounding. In prior work, we examined how structuring and problematizing scaffolding approaches can be instantiated through LLM agents in a scenario-based learning environment for diagnostic reasoning. While both approaches supported learning, we observed systematic differences in learner interaction patterns and clear tendencies indicating that different diagnostic strategies benefited from distinct forms of scaffolding. Building on these findings, we propose a theory-informed scaffolding design grounded in the Knowledge Learning Instruction (KLI) framework, as different diagnostic strategies target different types of knowledge and require different instructional mechanisms. We use KLI to guide the alignment between strategy demands and scaffolding approaches and introduce a KLI-informed hybrid LLM agent that adapts its pedagogical support according to the diagnostic strategy being practiced, rather than applying a single global scaffolding approach. We hypothesize that this design could enable better learning gains.