# What to Fix? Distinguishing between design and non-design rules in automated tools

## TL;DR

This paper empirically classifies software quality rules from industry tools into design and non-design categories to improve understanding and detection of technical debt related to design issues.

## Contribution

It provides a structured categorization of 466 rules from three tools, clarifying which rules target design problems versus syntactic quality.

## Key findings

- 55% of rules are non-design
- 19% of rules are design-related
- 26% of rules have labeling disagreements

## Abstract

Technical debt---design shortcuts taken to optimize for delivery speed---is a critical part of long-term software costs. Consequently, automatically detecting technical debt is a high priority for software practitioners. Software quality tool vendors have responded to this need by positioning their tools to detect and manage technical debt. While these tools bundle a number of rules, it is hard for users to understand which rules identify design issues, as opposed to syntactic quality. This is important, since previous studies have revealed the most significant technical debt is related to design issues. Other research has focused on comparing these tools on open source projects, but these comparisons have not looked at whether the rules were relevant to design. We conducted an empirical study using a structured categorization approach, and manually classify 466 software quality rules from three industry tools---CAST, SonarQube, and NDepend. We found that most of these rules were easily labeled as either not design (55%) or design (19%). The remainder (26%) resulted in disagreements among the labelers. Our results are a first step in formalizing a definition of a design rule, in order to support automatic detection.