Challenges in Comparing Code Maintainability across Different Programming Languages

TL;DR

This paper discusses the difficulties in comparing software maintainability across different programming languages, highlighting challenges related to paradigms, metrics, and quality dimensions based on 15 years of code analysis.

Contribution

It identifies key challenges in cross-language maintainability assessment and proposes considerations for producing comparable evaluations.

Findings

Challenges vary with programming paradigms.

Metrics need to be adapted for different languages.

Long-term analysis reveals complexity in maintainability comparison.

Abstract

Comparing the quality of software written in different computer languages is required in a variety of scenarios, e.g. multi-language projects or application selection process among candidates in different languages. We focus on the challenges related to comparing the maintainability quality typically through a maintainability index or technical debt approaches. We identify and discuss how to manage a number of challenges to produce comparable maintainability assessments across languages related to the programming paradigm (purely procedural vs OO vs multi-paradigm), the coverage of key quality dimensions, and the use of generic metrics vs more languages specific rules. Our work is based on a set of code analysis carried out in Wallonia over the past 15 years.