Do We Need Improved Code Quality Metrics?

TL;DR

This paper critically examines existing code quality metrics, identifies their deficiencies through developer surveys and case studies, and proposes improvements, especially for the LCOM metric, to better assess software cohesion.

Contribution

It provides a qualitative analysis of current metrics' shortcomings and introduces a new approach to improve the LCOM cohesion metric based on empirical evaluation.

Findings

Current metrics lack soundness and architecture assessment support.

Existing LCOM algorithms poorly represent cohesion in diverse cases.

Proposed LCOM approach aligns better with ground truth and real-world data.

Abstract

The software development community has been using code quality metrics for the last five decades. Despite their wide adoption, code quality metrics have attracted a fair share of criticism. In this paper, first, we carry out a qualitative exploration by surveying software developers to gauge their opinions about current practices and potential gaps with the present set of metrics. We identify deficiencies including lack of soundness, i.e., the ability of a metric to capture a notion accurately as promised by the metric, lack of support for assessing software architecture quality, and insufficient support for assessing software testing and infrastructure. In the second part of the paper, we focus on one specific code quality metric-LCOM as a case study to explore opportunities towards improved metrics. We evaluate existing LCOM algorithms qualitatively and quantitatively to observe how closely they represent the concept of cohesion. In this pursuit, we first create eight diverse cases that any LCOM algorithm must cover and obtain their cohesion levels by a set of experienced developers and consider them as a ground truth. We show that the present set of LCOM algorithms do poorly w.r.t. these cases. To bridge the identified gap, we propose a new approach to compute LCOM and evaluate the new approach with the ground truth. We also show, using a quantitative analysis using more than 90 thousand types belonging to 261 high-quality Java repositories, the present set of methods paint a very inaccurate and misleading picture of class cohesion. We conclude that the current code quality metrics in use suffer from various deficiencies, presenting ample opportunities for the research community to address the gaps.