Ambiguity Resolution with Human Feedback for Code Writing Tasks

TL;DR

This paper introduces ARHF, a system that identifies ambiguities in natural language code specifications, seeks human feedback to clarify these ambiguities, and generates code that resolves them, enhancing code writing accuracy.

Contribution

The paper presents a novel technique, ARHF, for ambiguity detection and resolution in code specifications using human feedback, improving code generation clarity.

Findings

ARHF effectively identifies ambiguous inputs in code specifications.

Human feedback significantly improves code accuracy and ambiguity resolution.

The system has potential educational benefits in teaching precise coding practices.

Abstract

Specifications for code writing tasks are usually expressed in natural language and may be ambiguous. Programmers must therefore develop the ability to recognize ambiguities in task specifications and resolve them by asking clarifying questions. We present and evaluate a prototype system, based on a novel technique (ARHF: Ambiguity Resolution with Human Feedback), that (1) suggests specific inputs on which a given task specification may be ambiguous, (2) seeks limited human feedback about the code's desired behavior on those inputs, and (3) uses this feedback to generate code that resolves these ambiguities. We evaluate the efficacy of our prototype, and we discuss the implications of such assistive systems on Computer Science education.