Metrics for Software Process Simulation Modeling

TL;DR

This paper systematically reviews and classifies metrics used in software process simulation modeling, providing an evidence-based reference to support process modelers and improve data collection and application.

Contribution

It offers a comprehensive classification of 2130 metrics from 145 studies and visualizes their causal relationships, enhancing understanding of metrics in SPS modeling.

Findings

Identified 2130 metrics across 145 studies.

Proposed diagrams illustrating causal relationships between metrics.

Provided strategies for data issues in SPS data preparation.

Abstract

Background: Software Process Simulation (SPS) has become an effective tool for software process management and improvement. However, its adoption in industry is less than what the research community expected due to the burden of measurement cost and the high demand for domain knowledge. The difficulty of extracting appropriate metrics with real data from process enactment is one of the great challenges. Objective: We aim to provide evidence-based support of the process metrics for software process (simulation) modeling. Method: A systematic literature review was performed by extending our previous review series to draw a comprehensive understanding of the metrics for process modeling following a meta-model of ontology of metrics in SPS. Results: We identified 145 process modeling studies that collectively involve 2130 metrics and classified them using the coding technique. Two diagrams which illustrate the high frequency causal relationships used between metrics are proposed in terms of two hierarchical levels of modeling purposes. We revisited the data issues encountered in SPS data preparing phases, as well as identified the corresponding strategies. Conclusion: The results of this study provide process modelers with an evidence-based reference of the identification and the use of metrics in SPS modeling, and further contribute to the development of the body of knowledge on software metrics in the context of process modeling. Furthermore, this study is not limited to process simulation but can be extended to software process modeling, in general. Taking simulation metrics as standards and references can further motivate and guide software developers to improve the collection, governance, and application of process data in practice.