Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding

TL;DR

This study explores how developers use 'vibe coding' with large language models, revealing diverse practices, challenges with AI's stochastic outputs, and implications for future software engineering tools and education.

Contribution

It provides the first qualitative analysis of vibe coding practices, highlighting behavioral spectra, mental models, and challenges faced by practitioners.

Findings

Vibe coders exhibit a range of engagement levels with AI-generated code.

Debugging often involves 'rolling the dice' due to AI's stochastic outputs.

Mental models influence prompting, evaluation, and trust in AI outputs.

Abstract

Large language models (LLMs) are reshaping software engineering by enabling "vibe coding," in which developers build software primarily through prompts rather than writing code. Although widely publicized as a productivity breakthrough, little is known about how practitioners actually define and engage in these practices. To shed light on this emerging phenomenon, we conducted a grounded theory study of 20 vibe-coding videos, including 7 live-streamed coding sessions (about 16 hours, 254 prompts) and 13 opinion videos (about 5 hours), supported by additional analysis of activity durations and prompt intents. Our findings reveal a spectrum of behaviors: some vibe coders rely almost entirely on AI without inspecting code, while others examine and adapt generated outputs. Across approaches, all must contend with the stochastic nature of generation, with debugging and refinement often described as "rolling the dice." Further, divergent mental models, shaped by vibe coders' expertise and reliance on AI, influence prompting strategies, evaluation practices, and levels of trust. These findings open new directions for research on the future of software engineering and point to practical opportunities for tool design and education.