Llama-Polya: Instruction Tuning for Large Language Model based on Polya's Problem-solving

TL;DR

This paper presents Llama-Polya, an instruction-tuned large language model that incorporates Polya's problem-solving framework to improve mathematical reasoning, pedagogical coherence, and metacognitive engagement in AI tutoring systems.

Contribution

It operationalizes Polya's problem-solving steps within an LLM, demonstrating improved reasoning structure and pedagogical alignment over traditional instruction tuning methods.

Findings

Enhanced reasoning-stage balance and fewer premature answers.

Improved pedagogical coherence and metacognitive prompting.

Limitations in personalization and mathematical rigor.

Abstract

This paper introduces Llama-Polya, an instruction-tuned large language model that integrates Polya's four-step problem-solving framework into its dialogue structure to support mathematical reasoning. Mathematical problem-solving is central to students' success in mathematics education, yet many learners struggle to plan, justify, and verify their solutions. Although large language models (LLMs) show promise as intelligent tutors, they often lack structured pedagogical alignment grounded in established learning theories. To address this gap, we operationalize Polya's problem-solving framework within an instruction-tuned LLM to promote metacognitive engagement and examine the effects of pedagogy-aligned fine-tuning compared to domain-only and general-purpose instruction tuning. Built on the Llama-3.1-8B architecture, Llama-Polya was fine-tuned on synthetic math problem-solving data derived from GSM8K, structured according to Polya's four stages. We developed and evaluated multiple variants-general-purpose instruct, math-domain metamath, pedagogy-aligned polya-v2, and sequential metamath+polya-v2-using both quantitative accuracy metrics and qualitative pedagogical assessments. Results indicate that models tuned with Polya's framework and domain-specific data produced more balanced reasoning-stage distributions and fewer premature answers. Expert evaluators also observed improved pedagogical coherence and metacognitive prompting, although limitations in personalization and mathematical rigor remained. These findings suggest that pedagogy-grounded instruction tuning can enhance educational alignment and reasoning transparency in LLM-based tutoring systems.