Characterizing the Experience of Subjects in Software Engineering Studies

TL;DR

This paper proposes a comprehensive scheme to characterize software engineering subjects' experience, validated through four empirical studies, highlighting the importance of multiple attributes for accurate profiling.

Contribution

It introduces an optimized scheme with three key attributes for characterizing subjects' experience in software engineering studies, validated across multiple empirical investigations.

Findings

Attributes are positively but moderately correlated.

Subjects often describe their experience by technical diversity.

The scheme is a feasible alternative for experience characterization.

Abstract

Context: Empirical studies in software engineering are typically centered on human subjects, ranging from novice to experienced developers. The experience of these individuals is a key context factor that should be properly characterized for supporting the design of empirical studies and interpreting their results. However, the criteria adopted for characterizing the experience of subjects do not follow a standard and are frequently limited. Goal: Our research aims at establishing an optimized and comprehensive scheme to characterize the subjects' experience for studies in software engineering. Method: Based on previous work, we defined the first version of this scheme, composed of three experience attributes, including time, number of projects, and self-perception. In the last years, we applied the characterization scheme over four empirical studies, reaching the characterization of 79 subjects in three different skills. Results: We found that the attributes from our scheme are positively but moderately correlated. This finding suggests these attributes play a complementary role in characterizing the subjects' experience. Besides, we found that study subjects tend to enumerate the technical diversity of their background when summarizing their professional experience. Conclusion: The scheme proposed represents a feasible alternative for characterizing subjects of empirical studies in the field. However, we intend to conduct additional investigations with developers to evolve it.