Developing Models of Procedural Skills using an AI-assisted Text-to-Model Approach

TL;DR

This paper presents a novel LLM-assisted text-to-model methodology that automates the creation of structured procedural knowledge models, significantly reducing expert effort and enabling scalable AI tutoring for procedural skills.

Contribution

The paper introduces a new TTM approach that automates the construction of TMK models using ontology-guided prompting, improving efficiency and scalability in AI tutoring system development.

Findings

Produced 23 TMK models for a graduate AI course

Reduced expert modeling time by 50-70%

Achieved high structural validity and reproducibility

Abstract

Scalable AI tutoring for procedural skill learning requires structured knowledge representations, yet constructing these representations remains a labor-intensive bottleneck. This paper introduces a new LLM-assisted text-to-model (TTM) methodology that transforms instructional materials into schema-complete Task-Method-Knowledge (TMK) models through ontology-constrained prompting and template-based generation, automating structural scaffolding while preserving expert oversight. Applied to a graduate-level online AI course, the methodology produced 23 TMK models - enabling full-course coverage for Ivy, a deployed AI coach that relies on TMK models to support learners' procedural understanding, for the first time. AI-assisted authoring reduced expert modeling time by 50-70% while producing structurally valid and highly reproducible models. We evaluate structural validity, semantic alignment, reproducibility, and refinement effort to characterize authoring scalability. Results indicate that the TTM methodology substantially lowers the cost of constructing structured procedural representations, making course-wide deployment of structured AI tutoring systems practically feasible.