Code Shaping: Iterative Code Editing with Free-form AI-Interpreted Sketching

TL;DR

This paper introduces code shaping, a novel interaction paradigm allowing programmers to use free-form sketches to communicate code edits to AI, supported by a user study and practical use cases.

Contribution

It presents the concept of code shaping, explores its interaction strategies through a user study, and demonstrates its application with real-world scenarios.

Findings

Different sketches serve various communication purposes.

Programmers employ specific strategies for iterative AI interactions.

Design principles facilitate effective reconciliation between sketches and code.

Abstract

We introduce the concept of code shaping, an interaction paradigm for editing code using free-form sketch annotations directly on top of the code and console output. To evaluate this concept, we conducted a three-stage design study with 18 different programmers to investigate how sketches can communicate intended code edits to an AI model for interpretation and execution. The results show how different sketches are used, the strategies programmers employ during iterative interactions with AI interpretations, and interaction design principles that support the reconciliation between the code editor and sketches. Finally, we demonstrate the practical application of the code shaping concept with two use case scenarios, illustrating design implications from the study.