Quality in model-driven engineering: a tertiary study

TL;DR

This tertiary study consolidates evidence on how model-driven engineering impacts software quality, highlighting maintainability as the most studied attribute and emphasizing the need for more empirical research in the field.

Contribution

It systematically reviews existing literature to map the main findings and identify research gaps regarding quality attributes affected by MDE.

Findings

Maintainability is the most studied quality attribute in MDE.

Most research questions are about mapping existing work, not empirical comparisons.

There is a broad coverage of software quality, but empirical validation is limited.

Abstract

Model-driven engineering (MDE) is believed to have a significant impact in software quality. However, researchers and practitioners may have a hard time locating consolidated evidence on this impact, as the available information is scattered in several different publications. Our goal is to aggregate consolidated findings on quality in MDE, facilitating the work of researchers and practitioners in learning about the coverage and main findings of existing work as well as identifying relatively unexplored niches of research that need further attention. We performed a tertiary study on quality in MDE, in order to gain a better understanding of its most prominent findings and existing challenges, as reported in the literature. We identified 22 systematic literature reviews and mapping studies and the most relevant quality attributes addressed by each of those studies, in the context of MDE. Maintainability is clearly the most often studied and reported quality attribute impacted by MDE. Eighty out of 83 research questions in the selected secondary studies have a structure that is more often associated with mapping existing research than with answering more concrete research questions (e.g., comparing two alternative MDE approaches with respect to their impact on a specific quality attribute). We briefly outline the main contributions of each of the selected literature reviews. In the collected studies, we observed a broad coverage of software product quality, although frequently accompanied by notes on how much more empirical research is needed to further validate existing claims. Relatively, little attention seems to be devoted to the impact of MDE on the quality in use of products developed using MDE.