Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education

TL;DR

This paper introduces TeachYou, an environment using large language models as teachable agents for programming education, employing a novel prompting pipeline to enhance knowledge-building and learner engagement.

Contribution

It presents a new prompting pipeline to control LLMs' knowledge, enabling effective teachable agents that promote active learning and misconception simulation.

Findings

Prompting pipeline effectively configures AlgoBo's problem-solving.

AlgoBo's questions lead to knowledge-dense conversations with effect size=0.71.

Design implications for cost-efficiency and personalization are discussed.

Abstract

This work investigates large language models (LLMs) as teachable agents for learning by teaching (LBT). LBT with teachable agents helps learners identify knowledge gaps and discover new knowledge. However, teachable agents require expensive programming of subject-specific knowledge. While LLMs as teachable agents can reduce the cost, LLMs' expansive knowledge as tutees discourages learners from teaching. We propose a prompting pipeline that restrains LLMs' knowledge and makes them initiate "why" and "how" questions for effective knowledge-building. We combined these techniques into TeachYou, an LBT environment for algorithm learning, and AlgoBo, an LLM-based tutee chatbot that can simulate misconceptions and unawareness prescribed in its knowledge state. Our technical evaluation confirmed that our prompting pipeline can effectively configure AlgoBo's problem-solving performance. Through a between-subject study with 40 algorithm novices, we also observed that AlgoBo's questions led to knowledge-dense conversations (effect size=0.71). Lastly, we discuss design implications, cost-efficiency, and personalization of LLM-based teachable agents.