Teaching Software Metrology: The Science of Measurement for Software Engineering

TL;DR

This paper emphasizes the importance of rigorous measurement in software engineering research, highlighting measurement challenges and proposing educational exercises to improve quantitative assessment practices.

Contribution

It introduces measurement principles from science to software engineering and provides practical exercises and datasets to enhance research quality.

Findings

Measurement issues are widespread in software engineering research.

Applying scientific measurement principles can improve research validity.

Educational tools can help researchers adopt better measurement practices.

Abstract

While the methodological rigor of computing research has improved considerably in the past two decades, quantitative software engineering research is hampered by immature measures and inattention to theory. Measurement-the principled assignment of numbers to phenomena-is intrinsically difficult because observation is predicated upon not only theoretical concepts but also the values and perspective of the research. Despite several previous attempts to raise awareness of more sophisticated approaches to measurement and the importance of quantitatively assessing reliability and validity, measurement issues continue to be widely ignored. The reasons are unknown, but differences in typical engineering and computer science graduate training programs (compared to psychology and management, for example) are involved. This chapter therefore reviews key concepts in the science of measurement and applies them to software engineering research. A series of exercises for applying important measurement concepts to the reader's research are included, and a sample dataset for the reader to try some of the statistical procedures mentioned is provided.