How to Measure Performance in Agile Software Development? A Mixed-Method Study

TL;DR

This study investigates the practical challenges of applying performance metrics in agile software development, highlighting issues like transparency and accuracy, and offers a repository of metrics and practical implications for improvement.

Contribution

It provides a comprehensive review of performance metrics used in agile development and insights into challenges faced in real-world application, along with practical recommendations.

Findings

Widely used metrics like story points and burn down charts face transparency and standardization issues.

Agile teams encounter challenges with metric accuracy and practical applicability.

The study offers a repository of performance metrics and practical guidance for practitioners.

Abstract

Context: Software process improvement (SPI) is known as a key for being successfull in software development. Measuring quality and performance is of high importance in agile software development as agile approaches focussing strongly on short-term success in dynamic markets. Even if software engineering research emphasizes the importance of performance metrics while using agile methods, the literature lacks on detail how to apply such metrics in practice and what challenges may occur while using them. Objective: The core objective of our study is to identify challenges that arise when using agile software development performance metrics in practice and how we can improve their successful application. Method: We decided to design a mixed-method study. First, we performed a rapid literature review to provide an up-to-date overview of used performance metrics. Second, we conducted a single case study using a focus group approach and qualitativ data collection and analysis in a real-world setting. Results: Our results show that while widely used performance metrics such as story points and burn down charts are widely used in practice, agile software development teams face challenges due to a lack of transparency and standardization as well as insufficient accuracy. Contributions: Based on our findings, we present a repository of widely used performance metrics for agile software development. Furthermore, we present implications for practitioners and researchers especially how to deal with challenges agile software development face while applying such metrics in practice.