Improving Procedural Skill Explanations via Constrained Generation: A Symbolic-LLM Hybrid Architecture

TL;DR

This paper introduces Ivy, a hybrid AI system that combines symbolic models with large language models to generate more structured and pedagogically valuable procedural explanations.

Contribution

It presents a novel hybrid architecture that constrains LLMs with symbolic task models to improve explanation quality in educational AI systems.

Findings

Symbolic constraints enhance explanation structure for 'how' and 'why' questions.

Ivy outperforms GPT baselines in explanation quality based on expert annotations.

Structured explanations are more pedagogically valuable.

Abstract

In procedural skill learning, instructional explanations must convey not just steps, but the causal, goal-directed, and compositional logic behind them. Large language models (LLMs) often produce fluent yet shallow responses that miss this structure. We present Ivy, an AI coaching system that delivers structured, multi-step explanations by combining symbolic Task-Method-Knowledge (TMK) models with a generative interpretation layer-an LLM that constructs explanations while being constrained by TMK structure. TMK encodes causal transitions, goal hierarchies, and problem decompositions, and guides the LLM within explicit structural bounds. We evaluate Ivy against responses against GPT and retrieval-augmented GPT baselines using expert and independent annotations across three inferential dimensions. Results show that symbolic constraints consistently improve the structural quality of explanations for "how" and "why" questions. This study demonstrates a scalable AI for education approach that strengthens the pedagogical value of AI-generated explanations in intelligent coaching systems.