Achievement Unlocked: A Case Study on Gamifying DevOps Practices in Industry

TL;DR

This large-scale industry study demonstrates that badge-based gamification can significantly accelerate DevOps practice adoption and positively influence key software delivery metrics, with mixed effects observed across different practices.

Contribution

The paper provides the first large-scale empirical evidence from industry on the effects of gamification in promoting DevOps practices and its impact on project metrics.

Findings

Accelerated adoption of DevOps practices by at least 60% with up to 6x increase in some cases

Teams earning testing badges increased bug fixing commits but reduced overall commits and pull requests

Teams with code review and quality badges improved delivery speed

Abstract

Gamification is the use of game elements such as points, leaderboards, and badges in a non-game context to encourage a desired behavior from individuals interacting with an environment. Recently, gamification has found its way into software engineering contexts as a means to promote certain activities to practitioners. Previous studies investigated the use of gamification to promote the adoption of a variety of tools and practices, however, these studies were either performed in an educational environment or in small to medium-sized teams of developers in the industry. We performed a large-scale mixed-methods study on the effects of badge-based gamification in promoting the adoption of DevOps practices in a very large company and evaluated how practice adoption is associated with changes in key delivery, quality, and throughput metrics of 333 software projects. We observed an accelerated adoption of some gamified DevOps practices by at least 60%, with increased adoption rates up to 6x. We found mixed results when associating badge adoption and metric changes: teams that earned testing badges showed an increase in bug fixing commits but output fewer commits and pull requests; teams that earned code review and quality tooling badges exhibited faster delivery metrics. Finally, our empirical study was supplemented by a survey with 45 developers where 73% of respondents found badges to be helpful for learning about and adopting new standardized practices. Our results contribute to the rich knowledge on gamification with a unique and important perspective from real industry practitioners.