Interactions with Prompt Problems: A New Way to Teach Programming with Large Language Models

TL;DR

This paper introduces Prompt Problems, a novel teaching approach where students craft prompts for LLMs to generate code, emphasizing understanding and translation skills over traditional coding exercises.

Contribution

It presents the design of a new educational tool that uses visual prompts and student-generated LLM prompts to teach programming concepts.

Findings

Students improved their understanding of problem translation.

The tool facilitated engagement with natural language programming.

Prompt Problems showed potential to complement traditional coding education.

Abstract

Large Language Models (LLMs) have upended decades of pedagogy in computing education. Students previously learned to code through \textit{writing} many small problems with less emphasis on code reading and comprehension. Recent research has shown that free code generation tools powered by LLMs can solve introductory programming problems presented in natural language with ease. In this paper, we propose a new way to teach programming with Prompt Problems. Students receive a problem visually, indicating how input should be transformed to output, and must translate that to a prompt for an LLM to decipher. The problem is considered correct when the code that is generated by the student prompt can pass all test cases. In this paper we present the design of this tool, discuss student interactions with it as they learn, and provide insights into this new class of programming problems as well as the design tools that integrate LLMs.