Grounded Copilot: How Programmers Interact with Code-Generating Models

TL;DR

This study analyzes how programmers interact with AI code assistants like Github Copilot, revealing a bimodal interaction pattern and offering design recommendations for future tools.

Contribution

First grounded theory analysis of programmer interactions with Copilot, identifying bimodal usage modes and informing usability improvements.

Findings

Interactions are bimodal: acceleration and exploration modes.

Programmers use Copilot to speed up known tasks and explore uncertain options.

Provides recommendations for enhancing AI assistant usability.

Abstract

Powered by recent advances in code-generating models, AI assistants like Github Copilot promise to change the face of programming forever. But what is this new face of programming? We present the first grounded theory analysis of how programmers interact with Copilot, based on observing 20 participants--with a range of prior experience using the assistant--as they solve diverse programming tasks across four languages. Our main finding is that interactions with programming assistants are bimodal: in acceleration mode, the programmer knows what to do next and uses Copilot to get there faster; in exploration mode, the programmer is unsure how to proceed and uses Copilot to explore their options. Based on our theory, we provide recommendations for improving the usability of future AI programming assistants.