Unifying Classification Schemes for Software Engineering Meta-Research

TL;DR

This paper aims to develop a unified classification scheme for software engineering meta-research by analyzing existing schemes, unifying their categories, and validating its applicability through empirical studies.

Contribution

It introduces a systematic approach to unify diverse SE research classifications into a single, generally applicable scheme, addressing a gap in meta-research documentation.

Findings

Unified classification scheme created and validated

Improved consistency and comparability of SE research classifications

Enhanced understanding of classification commonalities and differences

Abstract

Background: Classifications in meta-research enable researchers to cope with an increasing body of scientific knowledge. They provide a framework for, e.g., distinguishing methods, reports, reproducibility, and evaluation in a knowledge field as well as a common terminology. Both eases sharing, understanding and evolution of knowledge. In software engineering (SE), there are several classifications that describe the nature of SE research. Regarding the consolidation of the large body of classified knowledge in SE research, a generally applicable classification scheme is crucial. Moreover, the commonalities and differences among different classification schemes have rarely been studied. Due to the fact that classifications are documented textual, it is hard to catalog, reuse, and compare them. To the best of our knowledge, there is no research work so far that addresses documentation and systematic investigation of classifications in SE meta-research. Objective: We aim to construct a unified, generally applicable classification scheme for SE meta-research by collecting and documenting existing classification schemes and unifying their classes and categories. Method: Our execution plan is divided into three phases: construction, validation, and evaluation phase. For the construction phase, we perform a literature review to identify, collect, and analyze a set of established SE research classifications. In the validation phase, we analyze individual categories and classes of included papers. We use quantitative metrics from literature to conduct and assess the unification process to build a generally applicable classification scheme for SE research. Lastly, we investigate the applicability of the unified scheme. Therefore, we perform a workshop session followed by user studies w.r.t. investigations about reliability, correctness, and ease of use.