Automatic Large Language Models Creation of Interactive Learning Lessons

TL;DR

This paper presents an approach using GPT-4o and prompt engineering to automatically generate interactive, scenario-based lessons for training middle school math tutors, evaluated through human assessments.

Contribution

It introduces a task decomposition prompting strategy with GPT-4o for creating structured tutor training lessons, demonstrating improved quality over single-step methods.

Findings

Task decomposition improves lesson quality

Generated lessons are well-structured and save time

Limitations include generic feedback and clarity issues

Abstract

We explore the automatic generation of interactive, scenario-based lessons designed to train novice human tutors who teach middle school mathematics online. Employing prompt engineering through a Retrieval-Augmented Generation approach with GPT-4o, we developed a system capable of creating structured tutor training lessons. Our study generated lessons in English for three key topics: Encouraging Students' Independence, Encouraging Help-Seeking Behavior, and Turning on Cameras, using a task decomposition prompting strategy that breaks lesson generation into sub-tasks. The generated lessons were evaluated by two human evaluators, who provided both quantitative and qualitative evaluations using a comprehensive rubric informed by lesson design research. Results demonstrate that the task decomposition strategy led to higher-rated lessons compared to single-step generation. Human evaluators identified several strengths in the LLM-generated lessons, including well-structured content and time-saving potential, while also noting limitations such as generic feedback and a lack of clarity in some instructional sections. These findings underscore the potential of hybrid human-AI approaches for generating effective lessons in tutor training.