Towards Multi-Criteria Prioritization of Best Practices in Research Artifact Sharing

TL;DR

This paper proposes a framework using quality management principles and the Analytic Hierarchy Process to develop standardized guidelines for research artifact sharing, aiming to improve transparency and sustainability in software engineering.

Contribution

It introduces a novel combination of quality management and decision-making methodologies to establish common artifact evaluation guidelines in research.

Findings

Framework based on 5W2H principles for artifact evaluation

Use of Analytic Hierarchy Process for community-driven guideline development

Potential to enhance research software sustainability

Abstract

Research artifact sharing is known to strengthen the transparency of scientific studies. However, in the lack of common discipline-specific guidelines for artifacts evaluation, subjective and conflicting expectations may happen and threaten artifact quality. In this paper, we discuss our preliminary ideas for a framework based on quality management principles (5W2H) that can aid in the establishment of common guidelines for artifact evaluation and sharing. Also, using the Analytic Hierarchy Process, we discuss how research communities could join efforts to aid the guidelines' adequacy to research priorities. These combined methodologies constitute a novelty for software engineering research which can foster research software sustainability.