How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debugging

TL;DR

This paper presents HypoCompass, a system that uses LLMs as teachable agents to enhance programming debugging skills in students by enabling a collaborative learning environment focused on debugging and hypothesis generation.

Contribution

It introduces a novel system that leverages LLMs as teachable agents for debugging, improving training material quality and student debugging performance.

Findings

HypoCompass outperforms human counterparts fourfold in efficiency.

Students' debugging performance improved by 12% after using HypoCompass.

The system effectively delegates tasks between students and LLMs for better learning.

Abstract

Large Language Models (LLMs) now excel at generative skills and can create content at impeccable speeds. However, they are imperfect and still make various mistakes. In a Computer Science education context, as these models are widely recognized as "AI pair programmers," it becomes increasingly important to train students on evaluating and debugging the LLM-generated code. In this work, we introduce HypoCompass, a novel system to facilitate deliberate practice on debugging, where human novices play the role of Teaching Assistants and help LLM-powered teachable agents debug code. We enable effective task delegation between students and LLMs in this learning-by-teaching environment: students focus on hypothesizing the cause of code errors, while adjacent skills like code completion are offloaded to LLM-agents. Our evaluations demonstrate that HypoCompass generates high-quality training materials (e.g., bugs and fixes), outperforming human counterparts fourfold in efficiency, and significantly improves student performance on debugging by 12% in the pre-to-post test.