Sketch Then Generate: Providing Incremental User Feedback and Guiding LLM Code Generation through Language-Oriented Code Sketches

TL;DR

This paper introduces Language-Oriented Code Sketching, an interactive method that provides real-time, incremental feedback through code sketches during prompt creation, improving user guidance and interaction with LLMs for code generation.

Contribution

It presents a novel interactive approach that uses code sketches as intermediate feedback, leveraging NLP techniques to enhance prompt crafting for LLM-based code generation.

Findings

Provides instant, incremental feedback during prompt creation

Guides LLMs more effectively with code sketches

Enhances human-LLM interaction in coding tasks

Abstract

Crafting effective prompts for code generation or editing with Large Language Models (LLMs) is not an easy task. Particularly, the absence of immediate, stable feedback during prompt crafting hinders effective interaction, as users are left to mentally imagine possible outcomes until the code is generated. In response, we introduce Language-Oriented Code Sketching, an interactive approach that provides instant, incremental feedback in the form of code sketches (i.e., incomplete code outlines) during prompt crafting. This approach converts a prompt into a code sketch by leveraging the inherent linguistic structures within the prompt and applying classic natural language processing techniques. The sketch then serves as an intermediate placeholder that not only previews the intended code structure but also guides the LLM towards the desired code, thereby enhancing human-LLM interaction. We conclude by discussing the approach's applicability and future plans.