MetaCLASS: Metacognitive Coaching for Learning with Adaptive Self-regulation Support

TL;DR

MetaCLASS introduces a novel framework for metacognitive tutoring that supports self-regulated learning by planning pedagogical trajectories and generating dialogue, highlighting current limitations of LLMs in this domain.

Contribution

This paper presents MetaCLASS, a new framework for metacognitive tutoring that models move selection aligned with self-regulated learning, and provides a dataset and benchmarks for evaluating LLMs in this context.

Findings

Best LLM model achieves 43.2% accuracy in predicting tutor moves.

Models tend to over-intervene, rarely remaining silent when appropriate.

Traditional content-based models lack competence in metacognitive tutoring.

Abstract

Large language models can generate fluent explanations, but effective tutoring requires supporting the learner's thought process, not just delivering content. Metacognitive tutoring targets this gap by prompting planning, monitoring, debugging, and evaluation, and crucially, deciding when to be active versus minimally present, based on learner signals and trajectory. We introduce MetaCLASS, a learning-science grounded framework that formulates metacognitive tutoring as move selection over 11 interpretable actions aligned to self-regulated learning processes. MetaCLASS uses a two-phase framework that first plans a pedagogical trajectory conditioned on learner profiles (calibration, help-seeking) and then generates natural dialogue consistent with that plan. This yields a dataset of 1,015 conversations (7,711 turns) annotated with turn-level metacognitive labels, and validated for pedagogical contingency and trajectory adherence. We benchmark nine LLMs on predicting the next coach move given the problem and dialogue context. The best model achieves only 43.2\% accuracy, and models exhibit compulsive intervention bias: in turns where effective metacognitive tutoring requires silent (41.7\% of cases), models predict `no intervention' only 4.2\% of the time, while severely over-predicting high-intervention moves. These results show that traditional content-based tutoring ability does not translate to metacognitive tutoring competence, positioning MetaCLASS as a testbed for developing intelligent tutors that promote self-regulated learning.