PLAID: Supporting Computing Instructors to Identify Domain-Specific Programming Plans at Scale

TL;DR

This paper introduces PLAID, an LLM-powered tool that helps computing educators efficiently identify and structure programming plans, thereby supporting plan-focused teaching at scale.

Contribution

We developed and evaluated PLAID, a novel human-in-the-loop system that automates plan identification, reducing effort and increasing productivity for educators.

Findings

PLAID reduces cognitive demand for educators.

PLAID increases productivity in plan identification.

Educators find PLAID beneficial for instructional material creation.

Abstract

Pedagogical approaches focusing on stereotypical code solutions, known as programming plans, can increase problem-solving ability and motivate diverse learners. However, plan-focused pedagogies are rarely used beyond introductory programming. Our formative study (N=10 educators) showed that identifying plans is a tedious process. To advance plan-focused pedagogies in application-focused domains, we created an LLM-powered pipeline that automates the effortful parts of educators' plan identification process by providing use-case-driven program examples and candidate plans. In design workshops (N=7 educators), we identified design goals to maximize instructors' efficiency in plan identification by optimizing interaction with this LLM-generated content. Our resulting tool, PLAID, enables instructors to access a corpus of relevant programs to inspire plan identification, compare code snippets to assist plan refinement, and facilitates them in structuring code snippets into plans. We evaluated PLAID in a within-subjects user study (N=12 educators) and found that PLAID led to lower cognitive demand and increased productivity compared to the state-of-the-art. Educators found PLAID beneficial for generating instructional material. Thus, our findings suggest that human-in-the-loop approaches hold promise for supporting plan-focused pedagogies at scale.