Standards of Validity and the Validity of Standards in Behavioral Software Engineering Research: The Perspective of Psychological Test Theory

TL;DR

This paper advocates for adopting psychological test theory to improve validity assessment in behavioral software engineering research, emphasizing the importance of construct validity for more meaningful and reliable study outcomes.

Contribution

It introduces a psychological test theory-based approach to validity threats, enhancing the assessment of construct validity in behavioral software engineering studies.

Findings

Proposes a new approach to validity threats based on psychological test theory.

Highlights the importance of construct validity in behavioral software engineering.

Suggests broader application of validity principles to other software engineering research.

Abstract

Background. There are some publications in software engineering research that aim at guiding researchers in assessing validity threats to their studies. Still, many researchers fail to address many aspects of validity that are essential to quantitative research on human factors. Goal. This paper has the goal of triggering a change of mindset in what types of studies are the most valuable to the behavioral software engineering field, and also provide more details of what construct validity is. Method. The approach is based on psychological test theory and draws upon methods used in psychology in relation to construct validity. Results. In this paper, I suggest a different approach to validity threats than what is commonplace in behavioral software engineering research. Conclusions. While this paper focuses on behavioral software engineering, I believe other types of software engineering research might also benefit from an increased focus on construct validity.