Evolutionary Reinforcement Learning based AI tutor for Socratic Interdisciplinary Instruction

TL;DR

This paper introduces ERL4SIIP, an evolutionary reinforcement learning framework designed to enhance AI tutors for Socratic interdisciplinary STEM education by modeling student cognition and addressing reward sparsity.

Contribution

It formalizes the Socratic Interdisciplinary Instructional Problem as a POMDP and develops a novel ERL framework combining a student simulator, hierarchical rewards, and hybrid optimization strategies.

Findings

Effective modeling of student cognitive states.

Improved policy diversity and exploration.

Enhanced long-term educational goal achievement.

Abstract

Cultivating higher-order cognitive abilities -- such as knowledge integration, critical thinking, and creativity -- in modern STEM education necessitates a pedagogical shift from passive knowledge transmission to active Socratic construction. Although Large Language Models (LLMs) hold promise for STEM Interdisciplinary education, current methodologies employing Prompt Engineering (PE), Supervised Fine-tuning (SFT), or standard Reinforcement Learning (RL) often fall short of supporting this paradigm. Existing methods are hindered by three fundamental challenges: the inability to dynamically model latent student cognitive states; severe reward sparsity and delay inherent in long-term educational goals; and a tendency toward policy collapse lacking strategic diversity due to reliance on behavioral cloning. Recognizing the unobservability and dynamic complexity of these interactions, we formalize the Socratic Interdisciplinary Instructional Problem (SIIP) as a structured Partially Observable Markov Decision Process (POMDP), demanding simultaneous global exploration and fine-grained policy refinement. To this end, we propose ERL4SIIP, a novel Evolutionary Reinforcement Learning (ERL) framework specifically tailored for this domain. ERL4SIIP integrates: (1) a dynamic student simulator grounded in a STEM knowledge graph for latent state modeling; (2) a Hierarchical Reward Mechanism that decomposes long-horizon goals into dense signals; and (3) a LoRA-Division based optimization strategy coupling evolutionary algorithms for population-level global search with PPO for local gradient ascent.