Beyond the YAML File: Understanding Real-World GitHub Actions Workflow Adoption

TL;DR

This paper analyzes how developers adopt and respond to GitHub Actions workflows in real-world projects, revealing failure response patterns, usage correlations, and configuration-usage gaps through extensive data analysis.

Contribution

It provides the first large-scale quantitative and qualitative analysis of GitHub Actions adoption, identifying key patterns and hypotheses for future research.

Findings

Three failure response patterns identified

Higher GHA usage correlates with lower failure rates

Presence of configuration files can mask unused workflows

Abstract

Continuous Integration and Continuous Deployment (CI/CD) have become fundamental to modern software development, with GitHub Actions (GHA) emerging as a dominant automation platform. In this study, we analyze real-world execution records of GHA, examining how developers react to workflow failures, how these workflows are utilized by projects, and how these aspects relate to project characteristics. We quantitatively analyze 258,300 workflow run records from 952 repositories and perform an in-depth qualitative analysis of 21 selected, diverse GitHub repositories to understand how maintainers and contributors interact with workflow results. We identify three distinct failure response patterns, observe that higher usage intensity of GHA workflows correlates with lower failure rates, and uncover a configuration-usage gap where the presence of configuration files masks disabled or unused workflows. Moreover, our qualitative analysis of relationships between project characteristics and utilization patterns yields five hypotheses for future validation.