Automatic Categorization of GitHub Actions with Transformers and Few-shot Learning

TL;DR

This paper presents Gavel, a deep learning approach using Transformers and few-shot learning to automatically categorize GitHub Actions based on README files, improving visibility and organization.

Contribution

The work introduces Gavel, a novel Transformer-based method that effectively categorizes GitHub Actions, outperforming existing baseline approaches.

Findings

Gavel outperforms the state-of-the-art baseline in categorization accuracy.

Transformer models effectively leverage README content for classification.

The approach enhances the discoverability of GitHub Actions.

Abstract

In the GitHub ecosystem, workflows are used as an effective means to automate development tasks and to set up a Continuous Integration and Delivery (CI/CD pipeline). GitHub Actions (GHA) have been conceived to provide developers with a practical tool to create and maintain workflows, avoiding reinventing the wheel and cluttering the workflow with shell commands. Properly leveraging the power of GitHub Actions can facilitate the development processes, enhance collaboration, and significantly impact project outcomes. To expose actions to search engines, GitHub allows developers to assign them to one or more categories manually. These are used as an effective means to group actions sharing similar functionality. Nevertheless, while providing a practical way to execute workflows, many actions have unclear purposes, and sometimes they are not categorized. In this work, we bridge such a gap by conceptualizing Gavel, a practical solution to increasing the visibility of actions in GitHub. By leveraging the content of README.MD files for each action, we use Transformer--a deep learning algorithm--to assign suitable categories to the action. We conducted an empirical investigation and compared Gavel with a state-of-the-art baseline. The experimental results show that our proposed approach can assign categories to GitHub actions effectively, thus outperforming the state-of-the-art baseline.