A DSL for Defining Feature-Level Quality Constraints and the Aggregation of Evaluation Results in DevOps

TL;DR

This paper introduces a domain-specific language (DSL) for defining feature-specific quality constraints and aggregating evaluation results in DevOps, enabling tailored quality assessments based on feature importance and usage.

Contribution

The paper presents a novel DSL that operationalizes feature-level quality requirements, supporting automated evaluation and aggregation in DevOps environments.

Findings

The DSL allows precise definition of feature-specific quality constraints.

Automated evaluation of quality constraints is integrated into DevOps workflows.

An example application demonstrates the DSL's practical utility.

Abstract

Quality requirements typically differ among software features, e.g., due to different usage contexts of the features, different impacts of related quality deficiencies onto overall user satisfaction, or long-term plans of the developing organization. For instance, maintainability requirements might be particularly high for software features which are frequently used or bear strategic value for the developing organization. Also, software features where even the smallest delays are perceived as negative by the user will be subjected to specially tight performance requirements. We defined an operational DSL to define software quality requirements as individual feature-level constraints based on quantitative measures. The DSL provides language elements to define the operationalization of measures from external systems, time series operations, time filters, and the automatic evaluation of these feature-level constraints in DevOps based on comparison operators and threshold values. In addition, quality ratings summarize evaluation results of features on an ordinal grading scheme. Likewise, quality gates use these quality ratings to reflect the fitness of software features or the overall software product using different states. Finally, we show an example based on a widely-adopted secure mobile messaging app that illustrates the interplay of the different DSL elements.