Software Process Measurement and Related Challenges in Agile Software Development: A Multiple Case Study

TL;DR

This study investigates how Agile companies select and implement process metrics, revealing their motivations, challenges, and the use of custom metrics to improve development processes.

Contribution

It provides empirical insights into the rationale and operational challenges of metrics in Agile development through a multiple case study using the GQM approach.

Findings

Companies focus on velocity, testing, and estimation metrics.

Custom metrics are preferred for specific process assessments.

Operational challenges include data unavailability and technical constraints.

Abstract

Existing scientific literature highlights the importance of metrics in Agile Software Development (ASD). Still, empirical investigation into metrics in ASD is scarce, particularly in identifying the rationale and the operational challenges associated with metrics. Under the Q-Rapids project (Horizon 2020), we conducted a multiple case study at four Agile companies, using the Goal Question Metric (GQM) approach, to investigate the rationale explaining the choice of process metrics in ASD, and challenges faced in operationalizing them. Results reflect that companies are interested in assessing process aspects like velocity, testing performance, and estimation accuracy, and they prefer custom metrics for these assessments. Companies use metrics as a means to access and even capitalize on the data, erstwhile inaccessible due to technical or process constraints. However, development context of a company can hinder metrics operationalization, manifesting primarily as unavailability of the data required to measure metrics...(check the paper for full Abstract)