A Framework for Measuring the Quality of Infrastructure-as-Code Scripts

TL;DR

This paper introduces a new framework for assessing the quality of Infrastructure-as-Code scripts, demonstrated on Ansible repositories, revealing trends and providing a systematic tool for improvement.

Contribution

It presents a novel IaC code quality framework tailored for Ansible, enabling systematic assessment and fostering standardization in IaC script development.

Findings

Improvements in metadata and error handling over time

Declines in sophistication and automation in IaC scripts

Framework offers a foundation for continuous quality enhancement

Abstract

Infrastructure as Code (IaC) has become integral to modern software development, enabling automated and consistent configuration of computing environments. The rapid proliferation of IaC scripts has highlighted the need for better code quality assessment methods. This paper proposes a new IaC code quality framework specifically showcased for Ansible repositories as a foundation. By analyzing a comprehensive dataset of repositories from Ansible Galaxy, we applied our framework to evaluate code quality across multiple attributes. The analysis of our code quality metrics applied to Ansible Galaxy repositories reveal trends over time indicating improvements in areas such as metadata and error handling, while highlighting declines in others such as sophistication and automation. The framework offers practitioners a systematic tool for assessing and enhancing IaC scripts, fostering standardization and facilitating continuous improvement. It also provides a standardized foundation for further work into IaC code quality.