Convo: What does conversational programming need? An exploration of machine learning interface design

TL;DR

This paper explores how conversational interfaces can be optimized for programming tasks by comparing voice and text inputs, revealing user preferences and suggesting tailored, multimodal tools for different experience levels.

Contribution

It introduces a study comparing input methods for conversational programming, highlighting the importance of user experience and input mode flexibility in future tool design.

Findings

Users value different input modes for their strengths.

Novice users are more optimistic about voice programming.

Tailored interfaces improve user experience and efficiency.

Abstract

Vast improvements in natural language understanding and speech recognition have paved the way for conversational interaction with computers. While conversational agents have often been used for short goal-oriented dialog, we know little about agents for developing computer programs. To explore the utility of natural language for programming, we conducted a study ($n$=45) comparing different input methods to a conversational programming system we developed. Participants completed novice and advanced tasks using voice-based, text-based, and voice-or-text-based systems. We found that users appreciated aspects of each system (e.g., voice-input efficiency, text-input precision) and that novice users were more optimistic about programming using voice-input than advanced users. Our results show that future conversational programming tools should be tailored to users' programming experience and allow users to choose their preferred input mode. To reduce cognitive load, future interfaces can incorporate visualizations and possess custom natural language understanding and speech recognition models for programming.