Longitudinal Evaluation of Open-Source Software Maintainability

TL;DR

This longitudinal study analyzes the evolution of maintainability in open-source software over 18 years, comparing different models and examining development patterns and refactoring impacts.

Contribution

It provides a comprehensive comparison of maintainability models over a long period and offers insights into development practices affecting maintainability.

Findings

Maintainability varies significantly across versions.

Refactoring generally improves maintainability.

Different models have distinct strengths and weaknesses.

Abstract

We present a longitudinal study on the long-term evolution of maintainability in open-source software. Quality assessment remains at the forefront of both software research and practice, with many models and assessment methodologies proposed and used over time. Some of them helped create and shape standards such as ISO 9126 and 25010, which are well established today. Both describe software quality in terms of characteristics such as reliability, security or maintainability. An important body of research exists linking these characteristics with software metrics, and proposing ways to automate quality assessment by aggregating software metric values into higher-level quality models. We employ the Maintainability Index, technical debt ratio and a maintainability model based on the ARiSA Compendium. Our study covers the entire 18 year development history and all released versions for three complex, open-source applications. We determine the maintainability for each version using the proposed models, we compare obtained results and use manual source code examination to put them into context. We examine the common development patterns of the target applications and study the relation between refactoring and maintainability. Finally, we study the strengths and weaknesses of each maintainability model using manual source code examination as the baseline.