Developers and Generative AI: A Study of Self-Admitted Usage in Open Source Projects

TL;DR

This study investigates how open source developers self-report using generative AI tools like ChatGPT and GitHub Copilot, revealing expanded usage patterns and perceptions over two years through qualitative analysis.

Contribution

It introduces a taxonomy of 64 AI usage tasks in software development, based on manual coding of GitHub traces, extending prior work and providing insights into AI integration in workflows.

Findings

Usage avenues have expanded over two years.

Developers find generative AI tools useful in workflows.

Some concerns from previous years are no longer prevalent.

Abstract

The availability of generative Artificial Intelligence (AI) tools such as ChatGPT or GitHub Copilot is reshaping the way in which software is developed, evolved, and maintained. Oftentimes, developers leave traces of such an usage in software artifacts. This allows not only to understand how AI is used in software development, but also to let others be aware how such software artifacts were created, e.g., for licensing or trustworthiness purposes. This paper-building upon our preliminary work presented at MSR 2024-aims at qualitatively investigating on the self-admitted use of two very popular generative AI tools - ChatGPT and GitHub Copilot - in software development. To this aim, we mined GitHub for such traces, by looking at commits, issues and pull requests (PRs). Then, through a manual coding, we create a taxonomy of 64 different ChatGPT and GitHub Copilot usage tasks, grouped into 7 categories. By repeating our previous analysis two years after and by extending it to GitHub Copilot, we show how the usage avenues have been expanded, the extent to which developers perceived such a generative AI usage useful, and whether some concerns occurring more than one year ago are no longer present. The taxonomy of tasks we derived from such a qualitative study provided (i) developers with valuable insights into how generative AI can be integrated into their workflows, and (ii) researchers with a clear overview of tasks that developers perceive as well-suited for automation.