Software Code Quality Measurement: Implications from Metric Distributions

TL;DR

This paper introduces a distribution-based method for evaluating software code quality metrics, addressing the lack of standardization by analyzing a large dataset of open-source repositories to improve measurement consistency.

Contribution

It categorizes code quality metrics into monotonic and non-monotonic types and proposes a novel distribution-based scoring method for consistent evaluation.

Findings

Metrics demonstrate high explainability for software adoption

The method effectively evaluates both monotonic and non-monotonic metrics

Empirical analysis on 36,460 repositories validates the approach

Abstract

Software code quality is a construct with three dimensions: maintainability, reliability, and functionality. Although many firms have incorporated code quality metrics in their operations, evaluating these metrics still lacks consistent standards. We categorized distinct metrics into two types: 1) monotonic metrics that consistently influence code quality; and 2) non-monotonic metrics that lack a consistent relationship with code quality. To consistently evaluate them, we proposed a distribution-based method to get metric scores. Our empirical analysis includes 36,460 high-quality open-source software (OSS) repositories and their raw metrics from SonarQube and CK. The evaluated scores demonstrate great explainability on software adoption. Our work contributes to the multi-dimensional construct of code quality and its metric measurements, which provides practical implications for consistent measurements on both monotonic and non-monotonic metrics.