DevGPT: Studying Developer-ChatGPT Conversations

TL;DR

This paper presents DevGPT, a large dataset of developer-ChatGPT interactions, including prompts, responses, and related software artifacts, to facilitate research on AI-assisted programming and developer-LLM dynamics.

Contribution

The paper introduces a comprehensive, publicly available dataset of developer-ChatGPT conversations linked to software artifacts, enabling new research in AI-assisted software engineering.

Findings

Dataset includes 29,778 prompts and responses with code snippets.

Links interactions to source code, commits, issues, and discussions.

Facilitates analysis of ChatGPT's effectiveness in coding tasks.

Abstract

This paper introduces DevGPT, a dataset curated to explore how software developers interact with ChatGPT, a prominent large language model (LLM). The dataset encompasses 29,778 prompts and responses from ChatGPT, including 19,106 code snippets, and is linked to corresponding software development artifacts such as source code, commits, issues, pull requests, discussions, and Hacker News threads. This comprehensive dataset is derived from shared ChatGPT conversations collected from GitHub and Hacker News, providing a rich resource for understanding the dynamics of developer interactions with ChatGPT, the nature of their inquiries, and the impact of these interactions on their work. DevGPT enables the study of developer queries, the effectiveness of ChatGPT in code generation and problem solving, and the broader implications of AI-assisted programming. By providing this dataset, the paper paves the way for novel research avenues in software engineering, particularly in understanding and improving the use of LLMs like ChatGPT by developers.