BEAGLE: Behavior-Enforced Agent for Grounded Learner Emulation

TL;DR

BEAGLE is a neuro-symbolic framework that simulates student learning behaviors by integrating SRL theory, a semi-Markov model, Bayesian Knowledge Tracing, and a decoupled agent design, effectively mimicking authentic learner trajectories.

Contribution

It introduces a novel architecture that combines multiple technical innovations to address competency bias in LLM-based student simulation.

Findings

BEAGLE outperforms state-of-the-art baselines in reproducing authentic learning trajectories.

Participants in a Turing test could not reliably distinguish BEAGLE traces from real student data.

BEAGLE achieves statistically equivalent classification accuracy to chance in identifying simulated vs. real data.

Abstract

Simulating student learning behaviors in open-ended problem-solving environments holds potential for education research, from training adaptive tutoring systems to stress-testing pedagogical interventions. However, collecting authentic data is challenging due to privacy concerns and the high cost of longitudinal studies. While Large Language Models (LLMs) offer a promising path to student simulation, they suffer from competency bias, optimizing for efficient correctness rather than the erratic, iterative struggle characteristic of novice learners. We present BEAGLE, a neuro-symbolic framework that addresses this bias by incorporating Self-Regulated Learning (SRL) theory into a novel architecture. BEAGLE integrates three key technical innovations: (1) a semi-Markov model that governs the timing and transitions of cognitive behaviors and metacognitive behaviors; (2) Bayesian Knowledge Tracing with explicit flaw injection to enforce realistic knowledge gaps and "unknown unknowns"; and (3) a decoupled agent design that separates high-level strategy use from code generation actions to prevent the model from silently correcting its own intentional errors. In evaluations on Python programming tasks, BEAGLE significantly outperforms state-of-the-art baselines in reproducing authentic trajectories. In a human Turing test, participants could not reliably tell BEAGLE traces apart from real student data: classification accuracy was statistically equivalent to chance (52.8%, d' = 0.15, N = 71)