Rewarding How Models Think Pedagogically: Integrating Pedagogical Reasoning and Thinking Rewards for LLMs in Education

TL;DR

This paper introduces PedagogicalRL-Thinking, a framework that enhances LLMs for education by guiding their internal reasoning with domain-specific prompts and reinforcing pedagogical quality through specialized rewards, leading to improved educational performance.

Contribution

It presents two novel methods—pedagogical reasoning prompting and thinking reward—for aligning LLMs' internal reasoning with educational pedagogical principles.

Findings

Domain-specific prompting outperforms generic instructions.

Thinking reward combined with pedagogical prompting yields best results.

Models show improved reasoning and instructional decision-making.

Abstract

Large language models (LLMs) are increasingly deployed as intelligent tutoring systems, yet research on optimizing LLMs specifically for educational contexts remains limited. Recent works have proposed reinforcement learning approaches for training LLM tutors, but these methods focus solely on optimizing visible responses while neglecting the model's internal thinking process. We introduce PedagogicalRL-Thinking, a framework that extends pedagogical alignment to reasoning LLMs in education through two novel approaches: (1) Pedagogical Reasoning Prompting, which guides internal reasoning using domain-specific educational theory rather than generic instructions; and (2) Thinking Reward, which explicitly evaluates and reinforces the pedagogical quality of the model's reasoning traces. Our experiments reveal that domain-specific, theory-grounded prompting outperforms generic prompting, and that Thinking Reward is most effective when combined with pedagogical prompting. Furthermore, models trained only on mathematics tutoring dialogues show improved performance on educational benchmarks not seen during training, while preserving the base model's factual knowledge. Our quantitative and qualitative analyses reveal that pedagogical thinking reward produces systematic reasoning trace changes, with increased pedagogical reasoning and more structured instructional decision-making in the tutor's thinking process.