What is it like to program with artificial intelligence?
Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz,, Sruti Srinivasa Ragavan, Ben Zorn

TL;DR
This paper examines how large language models like Codex and AlphaCode are transforming programming, highlighting their unique properties, challenges, and implications for both expert and end-user programmers.
Contribution
It provides a comprehensive analysis of LLM-assisted programming, identifying its distinct characteristics and practical challenges compared to traditional programming assistance methods.
Findings
LLM-assisted programming shares similarities with compilation, pair programming, and search-based coding.
Fundamental differences exist in technical capabilities and user experience.
User study shows challenges faced by non-expert users in LLM-assisted tasks.
Abstract
Large language models, such as OpenAI's codex and Deepmind's AlphaCode, can generate code to solve a variety of problems expressed in natural language. This technology has already been commercialised in at least one widely-used programming editor extension: GitHub Copilot. In this paper, we explore how programming with large language models (LLM-assisted programming) is similar to, and differs from, prior conceptualisations of programmer assistance. We draw upon publicly available experience reports of LLM-assisted programming, as well as prior usability and design studies. We find that while LLM-assisted programming shares some properties of compilation, pair programming, and programming via search and reuse, there are fundamental differences both in the technical possibilities as well as the practical experience. Thus, LLM-assisted programming ought to be viewed as a new way of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpreadsheets and End-User Computing · Software Engineering Research · Cloud Computing and Resource Management
