Autonomy Is An Acquired Taste: Exploring Developer Preferences for GitHub Bots

TL;DR

This study investigates how developer preferences for GitHub bots vary based on bot behavior and developer experience, revealing a need for customizable bot configurations to suit individual and project needs.

Contribution

It provides empirical insights into developer preferences for bot autonomy and personality, highlighting the importance of configurable bots in collaborative software development.

Findings

Developers prefer personable, less autonomous bots.

Experienced developers favor more autonomous bots.

Configurable options are recommended for better alignment with preferences.

Abstract

Software bots fulfill an important role in collective software development, and their adoption by developers promises increased productivity. Past research has identified that bots that communicate too often can irritate developers, which affects the utility of the bot. However, it is not clear what other properties of human-bot collaboration affect developers' preferences, or what impact these properties might have. The main idea of this paper is to explore characteristics affecting developer preferences for interactions between humans and bots, in the context of GitHub pull requests. We carried out an exploratory sequential study with interviews and a subsequent vignette-based survey. We find developers generally prefer bots that are personable but show little autonomy, however, more experienced developers tend to prefer more autonomous bots. Based on this empirical evidence, we recommend bot developers increase configuration options for bots so that individual developers and projects can configure bots to best align with their own preferences and project cultures.