Intent Matters: Enhancing AI Tutoring with Fine-Grained Pedagogical Intent Annotation

TL;DR

This paper demonstrates that fine-grained pedagogical intent annotation significantly improves the alignment and effectiveness of LLM-generated tutoring responses in math education.

Contribution

It introduces a detailed taxonomy for annotating pedagogical intents and shows that fine-tuning LLMs with this data enhances educational response quality.

Findings

Fine-grained intent annotations lead to more pedagogically aligned responses.

Fine-tuned models outperform those trained on coarse taxonomy.

Annotated dataset and code are publicly released.

Abstract

Large language models (LLMs) hold great promise for educational applications, particularly in intelligent tutoring systems. However, effective tutoring requires alignment with pedagogical strategies - something current LLMs lack without task-specific adaptation. In this work, we explore whether fine-grained annotation of teacher intents can improve the quality of LLM-generated tutoring responses. We focus on MathDial, a dialog dataset for math instruction, and apply an automated annotation framework to re-annotate a portion of the dataset using a detailed taxonomy of eleven pedagogical intents. We then fine-tune an LLM using these new annotations and compare its performance to models trained on the original four-category taxonomy. Both automatic and qualitative evaluations show that the fine-grained model produces more pedagogically aligned and effective responses. Our findings highlight the value of intent specificity for controlled text generation in educational settings, and we release our annotated data and code to facilitate further research.